Hello everyone,
I have been using 3D Slicer for about ten years, but this is my very first post.
I am a digestive surgeon in France, and I built a module for my own use that I would like to share with you.
My goal is to be able to control my 3D models while I am operating. It is mainly useful in colorectal surgery (mostly right hemicolectomy), sometimes in gastric surgery, and for rare cases.
During my residency I tried to build something similar with an old Kinect, but with no coding skills at all I gave up.
I then used a Leap Motion a lot, but the company has changed hands, the device is no longer sold, and there is no longer any software support. It also emulated a mouse, so it was not very precise.
I still have no coding skills, but I used Claude to build it. HandsHover uses an ordinary webcam and Google’s MediaPipe hand tracking, running in a separate process so that Slicer stays responsive. It recognizes a few simple hand poses and drives Slicer’s views directly, without emulating a mouse. Almost everything is done with one hand:
- V sign + horizontal swipe: next / previous view (3D, Red, Green, Yellow)
- pinch: rotate the 3D model, or scroll through slices (faster gesture = faster scrolling)
- fist + up/down: zoom
- index finger + move: pan
- thumb up for 1 s: recenter the view
- two fists for 1 s: pause / resume, to avoid accidental commands
A banner at the top of the window always shows the active view, and camera images are processed locally on the computer: nothing is sent anywhere.
Here is the GitHub repository:
And here is a link to a short video made quickly (a rather busy weekend on call, and bad weather in the north of France, hence the lighting conditions
):
I will keep improving it over time, based on my own use and on your feedback, if it finds an audience.
Thank you for reading, and any feedback is very welcome!
Best regards,
Paul