Landmarking in virtual reality?

Support Feature requests
1

Hi,

I’m wondering whether there are plans to add further tools to the virtual reality module? Specifically, are there any plans to add landmarking capabilities to VR?

Thanks!
Dave

2

We (SlicerMorph) are not involved with virtual reality aspects of Slicer. So take it a with a grain of salt, I am not sure what the plans are.

Having said that, personally I am not convinced the virtual reality interfaces (not just in Slicer, pretty much everywhere) are sufficiently precise to do a good job for anatomical landmarking, particularly for quantitative morphology. I am having a hard time seeing mouse and keyboard (or digitization pads, pencils) being replaced for that purpose.

VR/AR is great of manipulation (e.g., sculpting a fossil reconstruction from fragments, possibly segmentation) and definitely for teaching and simulations. I am just not convinced it is the right tool for landmarking.

3

I think some work in that direction (VR/AR) has been already done Augmented reality guidance for burr hole placement using HoloLens - YouTube

1 Like
4

Thanks for your responses Murat and Leidy!

I see how it would be hard to match the precision of other methods. I thought that if it was possible, it would be fun to try it out and see how well it worked. I also thought that on complex 3D models (fish heads for both hard tissue and soft tissue contrast enhanced) the VR might make it substantially easier to place landmarks on internal bones since you have depth as an axis that can be controlled with the input device.

You mentioned segmentation as a use of VR. Do you know if that is currently possible in Slicer VR?

5

I do not know the latest state of virtual reality in Slicer. There is definitely some recent improvements. You can go to this page and see what’s possible currently. GitHub - KitwareMedical/SlicerVirtualReality: A Slicer extension that enables user to interact with a Slicer scene using virtual reality.

This is an example of interacting with segmented structures (but not necessarily segmenting them).

6

Since you can work at arbitrary scale in virtual reality, position objects directly (not with clunky mouse gestures) and the controllers are very accurate, I would expect that you can place landmarks with higher accuracy and less effort than with mouse and keyboard and a 2D screen.

However, markup placement in AR/VR in Slicer is not yet supported. We need to inplement snapping or laser-pointer like projection to allow placing points on surfaces. PerkLab is not currently working on this but Kitware or Ebatinca might.

7

It is possible. But it is all about the interface. I am happy to try and shown to be wrong, when it is available. In other VR applications, even with a laser-pointer like projection, I am finding localizing to the exact spot a lot more challenging with a controller that has 6 degrees of freedom in my shaky hand compared to a mouse moving on a flat surface.