Questions about Virtual Reality Implementation on Slicer

virtual-reality
#1

Hi, I have a few questions about Virtual Reality Implementation on Slicer

  1. I have a volume rendering that will not appear on Virtual Reality even though it appears on my 3D views. Why is this so and how do I fix it?
    Ex: Perspective from Slicer Desktop:
    image

Perspective from Slicer Virtual Reality:
image

  1. Is there any way to implement segmentation via Virtual Reality (ex: using the remote to segment)?

  2. I cannot see my Oculus touch controllers in Slicer, making it difficult to move around segmentations. Is there a setting I can change to make them visible?

0 Likes
(Csaba Pinter) #2

  1. I had issues like this in past versions (4.9) when the display node of the volume rendering did not have the VR view included (VR view node ID was not in ViewNodeIDs). Try to create the scene from scratch in 4.10.1.

  2. We want this a lot too. The main task that needs to be done first is to allow using Qt GUIs in VR in an easy way (see this ticket). We’re planning to work on this over the summer. Contributions are welcome! Let us know if you’re considering participating in this effort.

  3. The ControllerModelsVisible flag in vtkMRMLVirtualRealityViewNode controls the visibility. You can turn it on/off from the GUI using the “Controllers visible” checkbox in the Virtual Reality module. Again, please use 4.10.1.

I hope this helps!

0 Likes
#3

Thanks for the prompt reply.
2) I will continue to follow the development of the VR segmentation tool. Sounds very cool and would be very helpful!

In regards to 1) and 3), I am currently working on slicer 4.10.1 but I still do not see any virtual reality view of the controller models (after checking off the “Controllers visible checkbox”) nor the volume rendering, even after restarting the app and recreating the scene. I have also downloaded the latest nightly version of Slicer 4.11.0 and still do not see the volume rendering nor controller models. Am I missing a step?

0 Likes
(Andras Lasso) #4

Make sure you updated to the latest version of SlicerVirtualReality extension (you can check for updates and then install updates in the extension manager).

Controller display works fine for me for Windows Mixed Reality headsets and HTC Vive. I don’t have access to an Oculus Rift, but if for any reason you don’t see the controllers, you can always show where the controller is by enabling controller transforms and then applying the transforms to any models.

Volume rendering works fine for me for Windows Mixed Reality headsets and HTC Vive. Can you share an example scene that has volume rendering that does not show up in virtual reality?

0 Likes
#5

I have attached an example scene in gdocs:
https://drive.google.com/file/d/16FwaOCQseLVeWhtq9ytA9ofcI-g331ej/view?usp=sharing

How do I enable the controller transforms? They are visible on slicer as seen below:
image
, however turning on and off the view does nothing. If I apply the controller transform to the models I am still unable to see the controllers.

Thanks for the help.

0 Likes
(Andras Lasso) #6

In your scene, Volume Rendering is hidden in the virtual reality view. You can enable it in Volume rendering module / Inputs section / View:

image

As I wrote above, enable controller transforms (they appear in the scene, so you’ve done this correctly), but then don’t click on the eye icon of the transform (that would visualize the transform if you specify a region of interest), instead load any model (STL, OBJ, … file) that you would like to use as controller model and apply the controller transform to that.

By the way, is there a particular reason you chose an Oculus Rift headset? If you need high quality (absolute tracking, additional controllers, high resolution, etc.) then HTC Vive headsets are better, if you need something inexpensive or easily portable (no need for external trackers) then Windows Mixed Reality headsets are better.

1 Like
#7

Thank you so much for all the help! The reason I’m using Oculus is because I’m borrowing it temporarily from someone else to explore the virtual reality feature on slicer.

0 Likes
#8

Hi, sorry I had another question about the views. Is there a way through Python to automatically set views to include the “virtual reality view”? I’ve attempted to do it with getting the volume rendering node:
volumeRendering = slicer.mrmlScene.GetNodesByName(‘GPURayCastVolumeRendering’).GetItemAsObject(0)

and setting the view of the volume rendering node like so:
volumeRendering.SetViewNodeIDs(vrViewNodeID)

however, doing so does nothing. I’ve also noticed the vrViewNodeID is different from a regular view node ID (vtkMRMLVirtualRealityViewNode vs. vtkMRMLViewNode). Is there anyway to do this?

0 Likes
(Csaba Pinter) #9

This should not work, and you probably got an error as well. Instead, call AddViewNodeID

1 Like
#10

Thanks for the help,

Just wanted to write an update for this that I wrote a small work around for the “eraser effect” in Virtual Reality (in the meantime before you integrate the Qt GUIs in VR). Through this code, a sphere segmentation is created in VR that gets transformed to the right hand controller. Then, when I move the sphere around and it intersects with a part of a segmentation that needs to be erased (ie the spine) , I press a button on the keyboard that erases it (by using “subtract” in the “logical operators” effect). What I am wondering is that every time I press this button on the keyboard the Virtual Reality screen goes black for 1-3 seconds before an altered scene is shown. Is there anyway to fix this or is this inevitable as Slicer is running background processes?

Thanks!

0 Likes
(Andras Lasso) #11

The problem should go away if the segmentation is updated more quickly. For example, you can disable smoothing (checkbox in “Show 3D” button’s drop-down menu) to make 3D view updates magnitudes faster.

0 Likes