One of the most convenient way to use Slicer 3D in a surgery room is with a touchscreen/tablet.
From the existing posts and Youtube videos, it is possible to do so with a VNC free remote desktop software or just by running Slicer directly on a Windows tablet (ie Surface Pro).
About the later case, are there Slicer functionalities which cannot be run on a tablet (with no graphical card) ? What about segmentation and 3D reconstruction? Is there a part of preprocessing which you would advice to run before the surgery ?
I am speaking in term of software processing constraints and putting aside problems linked with DICOM loading and touch/mouse manipulation.
Slicer should run pretty well on a windows tablet. If there’s no GPU then you can only run software (CPU) volume rendering but that isn’t typically a big problem.
Using the user interface with touch controls can be difficult, so it can make sense to make a custom interface with big buttons as described here:
We’ve been using Slicer extensively in the operating room and interventional CT, MRI, and US imaging suites for various procedures during the last couple of years. Slicer is very flexible and it can run on even very modest tablets, so it is the clinical application and environment that determine what the best setup is:
what operations you need to perform (visualization only, or tool navigation, segmentation, registration, quantification)?
who operates the device: the surgeon or an engineer/computer scientist?
is the operator scrubbed in (need sterile interface)?
is there a display available for Slicer on the monitor boom that is used as the main display?
It is advisable to do everything that is possible before surgery to minimize operating room time and reduce chance of any unexpected difficulties.
If you give more information about the medical procedure and your environment (see questions above) then we can give more specific advice.
Yes I was thinking about running a slicelet for a custom, touchscreen-suitable and light application.
For now, I am still developping a scripted module for fast tests but you may hear about me and some of my questions in few months when I will start developping the slicelet.
@lassoan, to give you more details, here is what I can say about my procedure (but nothing is fixed yet) :
DICOM loading and preprocessing (segmentation, image registration and various other processing) will be made before the surgery. Ideally, we would like to let the surgeon do all operations on the same device (then a windows tablet)
During the surgery, the surgeon will use Slicer only for 3D model manipulation and landmarks translation. We don’t think yet a second monitor is necessary in the surgery room.
Yes for sure, maximum of manipulation and processing should be done before the surgery but again, we would like to run everything on one single device. We were thinking about having the heavy calculation run on the cloud or on a VPN free remote desktop but the use of a single device will make things easier.
But if you say that even modest tablet can run Slicer without too many problems, then segmentation and volume rendering shouldn’t be a problem for a Windows Surface Pro (i5 processor, 4-6Gb RAM).
During the surgery, we then need a sterile interface for the tablet, but we are thinking about specific plastic bags which allow touchscreen functionalities.
That’s a good start. You can transition seamlessly from a simple scripted module to a slicelet with custom user interface.
What landmarks do you need to translate and why? Is it for patient registration? Do you use surgical navigation (StealthStaion, BrainLab, etc.)?
A Microsoft Surface Pro can run Slicer very well, so it should not be a problem.
We use sterile X-ray cassette holder bags with Surface Pro. They are large enough so that it is easy to drop the tablet in them. We have the tablet in a rugged case, as it makes it easier to hold it without pressing the touch screen or turning it off (and may prevent the tablet from breaking if dropped). We disabled Windows button to prevent accidentally opening Start menu.
Even if Slicer runs on the tablet, it is important to set up a separate laptop with VNC access to that Slicer instance so that the engineer in the operating room can quickly check things, fix something, or help the surgeon if he/she does not know how to do something. During patient cases, it is typically not feasible for the engineer to explain anything to the surgeon.
About the landmarks, yes I am using them for bone registration. I am not using surgical navigation yet, but this could be of great help in the future. I may come back to the forum with questions about that later.
I understand your concern about the surgeon using Slicer on his/her own, but our goal is to build an easy-to-use application, even for non-experimented users. Of course, in the first times, additional tech or engineer will be here to help the surgeon, maybe on a separated computer with VPN access.
I am back after a long break and still working on Slicer. It is still a pleasure to use this software!
I received my Surface Pro tablet and I have a new question related to this post.
Is it possible to handle touch screen manipulation from a python script? For example, from the touch screen, it is possible to rotate the 3D view but I couldn’t find how to translate the view or apply a zoom. I think Slicer is not directly handling the touch screen, do I need to use mouse events ? Is this working for multi-touch recognition (i.e. 2 fingers to zoom or to translate)?
Slicer uses VTK for handling render window interactions and VTK currently does not support Windows multi-touch events. Using your fingers you can only rotate (touch-and-drag) and zoom (long-touch-and-drag) in 3D viewer; and adjust window/level and zoom in 2D viewer.
Ok, thanks for your answers. Zoom and rotation are then possible on 3D view, and I will display sliders or arrow buttons to let the user translate the view.
To inform other users, I am using a Windows Surface Pro tablet (i5, 8Gb of RAM) and Slicers is running perfectly, including volume rendering.