Sending Transformed Ultrasound Data to Unity for AR Overlay

Support
1

Hello,

Thank you for taking the time to read this post. I am currently working on visualizing ultrasound slice data through a semi-transparent mirror AR display and was hoping to get some help sending the data to Unity. I am currently using a SonixTouch Ultrasound machine with 2 6DOF NDI Aurora trackers. I have seen several resources showing me how to do the volume reconstruction of the US slices (haven’t completed this step yet), but I was wondering if there was any way to send the transformed data to Unity for further visualization (maybe through a Plus server?). Any help or assistance in this project would be greatly appreciated.

Thanks for taking the time to read this and I hope someone will be able to help me with my task!

2 Likes
2

Did you made any progress? Really curious in this topic as I am looking for the same application!

3

I think you may be able to do that on igtlink
@ungi may help

1 Like
4

Thanks for your response!
I am interested in using the Hololens as tracking device and sending it’s transformation into Slicer. Secondly, doing my registration procedure in Slicer and sending the resulting transformation to the Hololens/Unit for visualization thereafter. The user case is a tracked-US bone based registration using the Hololens.

To what extent is the work of @franklinwk applicable for my usercase?

5

I haven’t used HoloLens yet. But Slicer certainly can import and export transformations, images, points, string messages, etc. through OpenIGTLink with minimal latency. OpenIGTLink is a very simple communication protocol that can be added to any project without dependencies.

6

You can certainly build Unity applications that communicate with Slicer via OpenIGTLink. You can find code fragments in various projects for OpenIGTLink protocol implementation for Unity.

However, we would to move beyond Unity, because although it is very easy to get started with Unity, you need to reimplement lots of basic features. Instead, we are aiming for using OpenXR for rendering in Slicer directly to display in all kinds of compatible AR and VR headsets. The implementation is still experimental in VTK library and only some preliminary tests have been done in Slicer, but there is no readily usable solution yet. I would recommend to join the upcoming project week to learn more and collaborate with others on this topic.