From what I’ve seen, just putting the volume renderings or segmentations on the OR monitors can be difficult to interpret unless you’re able to move the model. Our current solution to this is to connect our laptop workstation (RTX 4070 with 64 GB RAM) to the DVI or HDMI port on the boom in the OR. The image can then be shown through as many displays as we would like in the OR, usually between 2 and 4. Movement of the model is done using a Leap Motion 2 controller, which allows you to manipulate the model and preserve sterility. What the 3D display does is minimize the amount of manipulation required to gain an understanding of the image.
I think we might be using these for different things. For us, AR overlay during open surgery may not add much because the structures are small in general, and you can usually identify the limits of solid organ tumors on palpation. The challenging open situations for us are ones in which there are complex networks of aberrant vessels within a tumor. The visual fidelity on the Hololens may not be enough to overlay multiple 2-3 mm vessels and follow with manipulations of the tumor about its axis (but I would be happy to be proven wrong).
For us the focus is on the preoperative planning phase or providing a roadmap for reference in the OR . The VR headsets provide good pictures for the first part but are not great for the second. The other consideration for us is that a lot of our complex cases are done under magnification with loupes on. Transitioning to the Hololens and back throughout the case would be cumbersome and might be difficult to maintain sterility. I’m curious how you and others have dealt with this in the past.
Do you have particular cases where you find this to be especially helpful? There is some overlap between adult and pediatric surgery but I’m always interested in finding new ways to improve how we do things. A lot of our complex cases are done under laparoscopy or robotically (choledochal cysts, anorectal malformations, etc). AR solutions that provide overlay data during laparoscopy or open with under 2.5-3.5x magnification would be ideal, but I don’t think those technologies exist yet. Thanks again for answering my questions. I’m still learning as I go here.