Extension for TiltBrush

Hello, I’m thinking of doing an extension that incorporates google’s tilt brush into Slicer. Has anyone done anything similar? Or thought of it?


Slicer can already display the scene and and interact with the nodes in virtual reality. It would be probably much easier to enable the Segment Editor paint tool work in virtual reality than integrating a completely new virtual reality framework.

That said, you probably don’t want to segment by free painting in 3D. Painting in 2D is already tedious and slow, and in 3D it would be much worse. However, a smoothing brush, or marker brush (to not paint new objects but mark suspicious regions for cleanup), or scissors could work well in virtual reality.


Yes we have been planning to integrate Segment Editor into SlicerVR for at least two years now (wow). The biggest obstacle currently is using SlicerVR with VTK9, which is far from trivial due to the major change in the way VTK9 handles external modules. Once that is solved, we need to do some work to be able to use a virtual user interface panel in VR, then integrating Segment Editor will be relatively easy (there are parts already done, but need to hammer out a few details).

Personally I am quite optimistic that we’ll have it done in a few months. It depends on two important things, however:

  1. Sorting out the build issue against VTK9. Maybe @jcfr can give an update as we have been in contact about this regularly, but not recently.
  2. The decision on a grant proposal in which the user interface panel is involved. We are supposed to hear about it “today, tomorrow, or early next week”.

I agree with @lassoan in that using a VR paintbrush is probably not a good idea to do segmentation from scratch because it is very hard to see the contours/borders even with a high quality volume rendering. Also agree that smoothing brush and touch-ups will be great. However, my suspicion is that we will only be able to assess the real advantages and best uses once we are actually able to try it. It will be quite a fun research and a good paper!