# VR extension with oculus rift

**URL:** <https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298>\
**Category:** Support\
**Created:** [May 4, 2023, 5:22pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298 "2023-05-04T17:22:53Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gonzalo\_Rojas\_Costa](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/gonzalo_rojas_costa/32/5259_2.png) [@Gonzalo\_Rojas\_Costa](https://discourse.slicer.org/u/Gonzalo_Rojas_Costa)\
**Post date:** [May 4, 2023, 5:22pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/1 "2023-05-04T17:22:53Z")

</div>

Hi:

I configure my oculus rift in my notebook. The games runs perfectly. Then, I install 3D slicer (version 5.2.2. in Windows) with VR extension, but when I select the VR glasses icon in 3D Slicer, in some seconds appears a red cross over it?. Why?

Sincerely,

Gonzalo Rojas Costa

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 4, 2023, 6:23pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/2 "2023-05-04T18:23:14Z")

</div>

Unfortunately, the VR extension of 3D Slicer is still broken. All my recent tests with the Oculus Quest2 failed with 5.x Slicer versions. The last working session I can remember was with 4.11.

---

<div class="post-metadata">

**Author:** ![Gonzalo\_Rojas\_Costa](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/gonzalo_rojas_costa/32/5259_2.png) [@Gonzalo\_Rojas\_Costa](https://discourse.slicer.org/u/Gonzalo_Rojas_Costa)\
**Post date:** [May 4, 2023, 7:59pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/3 "2023-05-04T19:59:16Z")

</div>

> [@rbumm](#):
>
> Unfortunately, the VR extension of 3D Slicer is still broken.

Do you know if exist any idea to upgrade it for 3D Slicer 5.x ?

---

<div class="post-metadata">

**Author:** ![cpinter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/cpinter/32/7995_2.png) [@cpinter](https://discourse.slicer.org/u/cpinter)\
**Post date:** [May 11, 2023, 10:33am UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/4 "2023-05-11T10:33:08Z")

</div>

@rbumm Can you please share detaills about how it is broken? It works for me in 5.3 except the two-handed gesture.

The one issue is you have to copy the manifest json files manually next to the Slicer executable. @jcfr I’m not sure, do we have some guidelines about it or any way to make this easy for the user?

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 11, 2023, 12:11pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/5 "2023-05-11T12:11:27Z")

</div>

That’s correct! The Oculus Quest 2’s right-handed joystick allows you to manipulate objects in 3D space. By using the joystick, you can move the object around, and you can adjust the direction of movement by tilting the right controller.

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/0/d/0dc33c5b880a6e867c9e7f8f6e443355eb17b11d.jpeg)

To ensure proper functionality, one must make sure that the manifest JSON files are located in the same directory as the Slicer.exe file.

In version 4.11, you could grab an object by simultaneously pressing both grab triggers, and then rotate and tilt the object as desired. However, this feature is currently broken in version 5. When attempting this action, the object disappears from the 3D space.

---

<div class="post-metadata">

**Author:** ![cpinter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/cpinter/32/7995_2.png) [@cpinter](https://discourse.slicer.org/u/cpinter)\
**Post date:** [May 11, 2023, 12:19pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/6 "2023-05-11T12:19:22Z")

</div>

> [@rbumm](#):
>
> In version 4.11, you could grab an object by simultaneously pressing both grab triggers, and then rotate and tilt the object as desired.

What you could grab in 4.11 was the “world” and not the object, this is an important difference from what you say. So the object stayed the same in RAS, but the camera and the scaling of the world changed (just try it with multiple objects in the scene and you’ll see).  
That said, I miss this feature very much, because it is super convenient to navigate that way. I don’t have funding for this, and unfortunately also don’t have free time these months, so the same as you, I can only wait. However, I’m trying to be present in the efforts that do have funding (although it is targeted only tangentially because it’s for AR), and whenever I see that I have a day to work on this I’ll coordinate with them and get this done finally.

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 11, 2023, 12:29pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/7 "2023-05-11T12:29:30Z")

</div>

Ok, I understand. Do you know where the camera position adjustment is made in the Slicer VR source code and which VTK function would be responsible for moving the camera?

---

<div class="post-metadata">

**Author:** ![cpinter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/cpinter/32/7995_2.png) [@cpinter](https://discourse.slicer.org/u/cpinter)\
**Post date:** [May 11, 2023, 12:51pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/8 "2023-05-11T12:51:27Z")

</div>

You can find the details of our findings on this page, and at the links pointed from it: [ProjectWeek/README.md at master · NA-MIC/ProjectWeek · GitHub](https://github.com/NA-MIC/ProjectWeek/blob/master/PW38_2023_GranCanaria/Projects/SlicerVRInteractions/README.md)

Basically where this two-handed gesture is managed is the `RecognizeComplexGesture` function (mentioned in some of the pull requests referenced). The setting of the camera transform as I remember was in the VR interactor style class. If you want further details please don’t hesitate to ask.

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 11, 2023, 2:08pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/9 "2023-05-11T14:08:48Z")

</div>

From a clinical point of view it would be absolutely great to have a ROI based crop tool for volume renderings and segmentation in VR.

---

<div class="post-metadata">

**Author:** ![Gonzalo\_Rojas\_Costa](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/gonzalo_rojas_costa/32/5259_2.png) [@Gonzalo\_Rojas\_Costa](https://discourse.slicer.org/u/Gonzalo_Rojas_Costa)\
**Post date:** [May 12, 2023, 4:45am UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/10 "2023-05-12T04:45:22Z")

</div>

You used 5.3 version with which glasses? Oculus Rift? Because we are trying to configure our Oculus Rift with 5.3 version, and it doesn’t works.

---

<div class="post-metadata">

**Author:** ![cpinter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/cpinter/32/7995_2.png) [@cpinter](https://discourse.slicer.org/u/cpinter)\
**Post date:** [May 12, 2023, 8:41am UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/11 "2023-05-12T08:41:20Z")

</div>

I don’t have an Oculus Rift anymore, I tried HTC Vive Pro and HP Reverb 2. I still don’t know what it means for you exactly that “it doesn’t work”. Some detailed information, hopefully screenshots would be useful.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [May 13, 2023, 4:36pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/12 "2023-05-13T16:36:48Z")

</div>

> one must make sure that the manifest JSON files are located in the same directory as the Slicer.exe file.

The associated issue \[1\] has been fixed\[2\] in the `SlicerVirtualReality` project.

To support configuring the the folder in which the action manifest files are located, the relevant fix from upstream VTK were integrated in our Slicer/VTK fork\[3\].

**Question:** Could someone try to download the latest preview build of Slicer and try the SlicerVirtualReality extension ?

* * *

1. [https://github.com/KitwareMedical/SlicerVirtualReality/issues/112](https://github.com/KitwareMedical/SlicerVirtualReality/issues/112) 

2. [https://github.com/KitwareMedical/SlicerVirtualReality/pull/114](https://github.com/KitwareMedical/SlicerVirtualReality/pull/114) 

3. [https://github.com/Slicer/Slicer/pull/6967](https://github.com/Slicer/Slicer/pull/6967)

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 13, 2023, 6:50pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/13 "2023-05-13T18:50:24Z")

</div>

Downloaded the latest preview build of Slicer and installed the SlicerVirtualReality extension.  
With or without the JSON files the segmentation is shown in the headset, but none of the keys for manipulation work (joystick, grab etc) But grabbing with both controllers does not let the object dissapear from the scene.  
This was tested with the Quest 2 (it shows a Rift connection) but on an older PC with a GTX 970, still in quite good shape, I will try 4.11 next just to make sure the system generally works.

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 13, 2023, 7:10pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/14 "2023-05-13T19:10:22Z")

</div>

4.11 works well on that old system and has all interaction keys active (joystick and both grab triggers).Sidenote: Exiting VR (SteamVR) crashes Slicer.

---

<div class="post-metadata">

**Author:** ![cpinter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/cpinter/32/7995_2.png) [@cpinter](https://discourse.slicer.org/u/cpinter)\
**Post date:** [May 15, 2023, 2:17pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/15 "2023-05-15T14:17:33Z")

</div>

Same here with the latest preview. Without copying the jsons I get this, and no interactions work. I didn’t try copying the file as I understand the goal was to check if it works without manual copying.

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/3/3/33d6c05fc6dc05cb071e396c3de607c6384e1623.jpeg)

Thanks for working on this @jcfr !

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 16, 2023, 11:41pm UTC](https://discourse.slicer.org/t/vr-extension-with-oculus-rift/29298/16 "2023-05-16T23:41:33Z")

</div>

Interestingly, in my case, I did not get this missing action manifest message in SteamVR this time without the JSONS. On a second try with the JSONS, at least one of the buttons seemed to work (the “B” left controller menu button).