Adding medical tools in segment editor and perform erase effect!

Development
1

Hi Experts,

In our project, instead of sphere brush (default), we would like to import our own medical tools (obj) file and make an erase effect on 3D model from that medical tool. What will be the fastest and convenient way to do so.

Please provide some guidance .

Thank you.

2

@lassoan, By the way, we already have Virtual Reality version for this simulator, we would like to implement it also in the PC version. My current plan is to load the obj file (medical tool) and transform it to the sphere tool of the segment editor.

Secondly,
Is the link primary source code which is controlling paint and erase effect of 3D Slicer ?

Please feel free to write.

Thank you,

Sincerely
Raj

3

Segment Editor is not ideal for real-time simulation of drilling. For real-time volume removal, you are probably better off using volume rendering for 3D display and blanking out parts of the volume by combining your tool volume (rasterized volume of the tool model) with the displayed volume. This should be all doable in Python and since you can do everything with numpy array operations, it should be very fast.

4

Hi @lassoan , thank you for reply. Actually our spine models are built/resected with uploading patient specific MR/CT scans and using Segment Editor module.

Yes for our Virtual Reality system there is huge latency during the resection and we tried to use volume rendering technique instead of using closed surface representation. The study is still going on.

In addition with VR simulation, we do have regular PC version simulation, right now we are just using 3D sphere or scissor effect for resection. Though it doesn’t look real time but is acceptable for training purpose.

Our plan is instead of using 3d sphere or brush for erase effect, we planned to import medical tools like burr/ Kerrison as obj file and make a erase effect.

(Slicer/PaintEffect.py at 0daa3b92680706c1e98b6585a676545f41c0b8a2 · Slicer/Slicer · GitHub)

I was trying modify and use the painteffect.py code from github but it seems there is no any effects of this code in the paint/erase in 3d slicer.

Please suggest for our approach.

Thank you !!!

5

Hi @lassoan , @pieper , @muratmaga ,

Any suggestions, How can I use 3D obj file model as a brush in erase effect instead of 3D sphere.

Thank you !!

Sincerely,
Raj

6

You can apply a transform to your 3D model, import it into the segmentation, and erase from another segment using Logical operators effect. You can find examples for each of these steps in the script repository.

7

Hi @lassoan

Thank you for the suggestions. Only one thing, I got stuck is how to get the node of the 3D sphere being used in paint/erase effect before apply transform to imported 3D model.

Thank you

8

See this example for importing a loading a 3D model and importing it into segmentation: https://slicer.readthedocs.io/en/latest/developer_guide/script_repository.html#rasterize-a-model-and-save-it-to-a-series-of-image-files

9

Hi @lassoan , Thank you for the scripts.

Last question, I am still struggling . In segment Editor 3D sphere brush is used for paint/erase effect. How can I access that 3D sphere Brush node so that I can transform my loaded obj to that model node.

Thank you

10

You can use Logical operators effect to modify (paint or erase) another segment using your imported segment.

11

Thank you @lassoan ,

But there should be user-interactions with mouse to the imported model during erase as available in current paint effect .

12

You can implement custom user interactions in your module (add observers to mouse events, etc. or create custom effect, using qSlicerSegmentEditorPaintEffect as base class and customizing the updateBrushModel method). I would not recommend doing this, though. Displaying the bone using volume rendering and blanking out regions that are being removed results in much higher quality rendering and a magnitude higher frame rate.