Improved interactions in virtual reality

New features in SlicerVirtualReality:

  • Position MRML objects in VR
  • Two-handed world manipulation gesture for pan, rotate, zoom

The video shows the improvements used for pedicle screw placement procedure with three visualization techniques:

  • Volume rendering with dynamic slice view
  • Solid volume rendering
  • Surface rendering

Available for 3D Slicer version 4.10

3 Likes

Looks great!

A couple questions come to mind:

  • watching the video, it appears you are very confidently just putting the screw in the right place. Compared to doing the same operation with the mouse, how does the interaction feel in terms of accuracy and efficiency?

  • do you get ‘gorilla arm’ with that style of interaction?

It was more like putting it to approximately correct position in 5-10 seconds then spend tens of seconds to look at it from various directions and fine-tune the position.

I didn’t experience any arm fatigue when I tried it. I always moved/rotated/zoomed the world to be close to me (it is quite intuitive now with the two-handed grab gestures) and so I could keep my arms close to my body, in a comfortably bent position. Csaba does it similarly in the video - he reaches out, pulls the model to be right in front of him, and then the rest is mostly fine manipulation. Also, I always tried this only for couple of minutes at a time, so maybe this was just not long enough to get tired.

@cpinter did you have the same experience?

1 Like

Thanks for the info. Sounds like a NASA-TLX style of study would be interesting.

It’s very easy to see in stereo the approximate position it needs to be, so it’s basically one hand motion to move it there. But then as the canal where the screw needs to be is quite narrow, a lot of small adjustments are needed. And for that clipping is super useful. Looking forward to the sphere clipping you guys prototyped in Las Palmas!

I have worked for weeks on the controller interactions, which means I was basically continually waving my arms. I experienced no fatigue. As @lassoan says it is intuitive to pull the objects close to me for better visibility, and in that position I can even put my elbows on the armrest of my chair sometimes and then it becomes effortless. I’d argue that it’s more ergonomic than a regular mouse, for which one has to hold their wrists in an unnatural angle.
I can imagine fatigue if you have to do it for half an hour without interruption and without supporting your elbows, but I don’t see that with our current applications.

1 Like