Slicer to Magic Leap Integration

Development
,
#1

Hello,
I am exploring ways to integrate Slicer 3D.

I am looking to use SlicerVirtualReality. Can someone help me with file format that gets IN and Out of SlicerVirtualReality? This will be helpful to explore its compatibility with Magic leap.

Thanks

#2

As the MagicLeap is not an OpenVR supported device, it cannot be used currently with Slicer Virtual Reality. We are exploring the possibility of moving to OpenXR, but are looking for help in this area.

#3

Thank @adamrankin . I am looking to use this integration for my research. I am open for collaboration. Please let me know if we can work together on exploring this.

#4

@adamrankin , can you let me know the file format that goes IN and Out of SlicerVirtualReality? Is it a video stream?

#5

I’m not quite sure I follow. VR just shows you your current 3D scene in a VR environment. Anything you have in your 3D view on your desktop, you’ll see in VR.

#6

yes, but there is communication between slicer and VR environment through SlicerVirtualReality plugin. Thats the reason the 3D scene in slicer (desktop), shows up in VR. I am trying to understand what kind of communication is that?

#7

@cshreyas VTKRenderingOpenVR has a special VTK render window that renders two scenes that are transferred by OpenVR to the headset.

@adamrankin We are waiting for the result of a grant in which we included OpenXR integration. We should get word within a few weeks (they promised by the end of March but who knows).

1 Like
#8

VTK developers had plans for working on OpenXR integration, too. It may worth asking on the VTK forum for latest updates.

2 Likes
#9

Thanks @lassoan, @cpinter . Of what I heard from Magic Leap support, they have no plans to support OpenXR in near future.

I am trying to use OpenIGTLink as a bridge between Slicer and Magic Leap. This would have to do be done using OpenCV most likely and publish the point cloud data to a streaming server.

Do you have any pointers on where to get started from the slicer code base? Where are the images getting published in Slicer?

#10

You may consider moving to a device that has more certain future and supports OpenXR, such as Microsoft Hololens.

What data do you plan to send between Slicer and Magic Leap?

Could you clarify? What is the role of OpenCV? What data you would like to get from/send to Magic Leap?

All incoming data (images, transform, surface meshes, etc.) appear as MRML nodes. If you set a MRML node as outgoing node then any changes in that node are automatically sent via OpenIGTLink.