Creating a basic Paint Foreground and Paint Background Tool

Support
1

Hi,
I am new to 3D Slicer.
I am trying to implement an interactive segmentation tool for correcting issues in existing segmentation labels. The expected behaviour of this tool is to have “background” and “foreground” paint brushes, so any issues in existing segmentation labels can be corrected by painting over the viewed data.

What I intend to implement is a basic extension that provides two buttons (one for background and another for foreground). When foreground is selected, the paint tool should paint in a specific color (e.g. green). For background the paint tool should paint a different color (e.g. red). In the python script I will access the masks for each and save for corrected label processing through a logic.

I have looked around and into the Segment Editor Widget. However I am unsure about what is the best way to implement a simple gui that can allow user to switch between foreground/background paints (and corresponding segmentation layers), while still providing the ability to click and interact with the data.

I would be very grateful if you can provide with any material that can help me in implementing the above flow. Many thanks!

2

Hi -

If I understand your goal right it may already be available via the ‘w’ key, that cycles through the segments.

https://slicer.readthedocs.io/en/latest/user_guide/modules/segmenteditor.html?highlight=shortcuts#keyboard-shortcuts

3

For correcting segmentations, you may also find useful the “Color smudge” option in Paint effect. When enabled, clicking on any segment and dragging it into an empty region or into another segment will overwrite that region with the selected segment.

4

Hi Andras,
Many thanks! Do you have any guidance/example on how I can include just the two tools (Paint and Smudge) in my own extension? As I will always be painting foreground and background, it be would be great to hide this in extension and always initialize to have foreground/background layers to paint in. I need to implement this as a separate extension from the Segmentation Effect extension as the goal of my project is to eventually send the painted regions to a research framework (which will be used to correct the original labels). However as a starting point I need to have this in an extension and have the foreground and background paint accessible so we can then send them to the research algorithm.

5

Hi Pieper,
Many thanks for providing this. Is there a way I can include the “Paint” (or selected effects) only in my own extension? I need to implement a separate extension from Segment Editor Effects as this user interaction then needs to be send to a research framework for correcting the labels. I would appreciate if you can point me to a simple example that uses Segment Editor widget in similar application.

6

You can put a segment editor widget into your own module GUI and this widget is highly customizable. You can choose which effects to show, etc. If you really want just a smudge brush then probably the best is to hide the segment editor widget and just put a few buttons on your module GUI that control the hidden widget.

7

Many thanks Andras. This is very helpful for me.

I understand now how to do this. Using the example from here: This example demonstrates how to do segment brain tumor using Nvidia's AI-assisted annotation tool in batch mode (without GUI, using qMRMLSegmentEditorWidget) using 3D Slicer · GitHub, I have changed it to allow painting and adding two segmentation layers.

# Create segmentation node to store the segmentation result
segmentationNode = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSegmentationNode")
segmentationNode.CreateDefaultDisplayNodes()
# add foreground
segmentationNode.GetSegmentation().AddEmptySegment()
# add background
segmentationNode.GetSegmentation().AddEmptySegment()

# Create segment editor to get access to effects
segmentEditorWidget = slicer.qMRMLSegmentEditorWidget()
segmentEditorWidget.setMRMLScene(slicer.mrmlScene)
segmentEditorNode = slicer.vtkMRMLSegmentEditorNode()
slicer.mrmlScene.AddNode(segmentEditorNode)
segmentEditorWidget.setMRMLSegmentEditorNode(segmentEditorNode)
segmentEditorWidget.setSegmentationNode(segmentationNode)
segmentEditorWidget.setMasterVolumeNode(masterVolumeNode)

segmentEditorWidget.setActiveEffectByName("Paint")
effect = segmentEditorWidget.activeEffect()

I have three questions regarding this. How can I switch between the two layers? Is there a function call to do the same as pressing ‘w’ in the Segmentation Editor?
Secondly, how I can change the size of Paint brush?
Third, once I am done - how can I extract the foreground and background masks?

1 Like
8

You can use setCurrentSegmentID method.

See parameters of paint effect here.

You can convert segmentation to various other nodes as shown in examples in the script repository.

1 Like