Segment Editor Effect: Fill Between Slices does not recognize existing contour

Development
1

Operating system: Windows 10
Slicer version: 4.8.1 (r26813) built myself

Background:
I have started some work on automatic segmentation using the Segment Editor framework. Now I have an automatically generated contour for the spinal canal. The contour is present in some slices (about every 3rd), and is not present in others. Now I thought about including some methodology from “Fill between slices” module. The first attempt before automating is testing manually by clicking through the GUI.

Approach - step by step

  1. Select contour called “Spinal canal” and open “Fill between slices” module
  2. Click “Initialize”
  3. Check the result by using the slider - the results contour is empty
  4. Then I selected the “Paint” module for some testing, draw a small region in an empty slice of “Spinal canal” contour.
  5. Now try “Initialize” again in the “Fill between slices” module, and wow, the result looks good.

Problem analysis
Obviously, there’s something wrong which makes my automatically segmented structure not being recognized by the “Fill between slices” module. But what?

I analyzed the source and had a look at the file AbstractScriptedSegmentEditorAutoCompleteEffect.py, and the preview method, there around line 342, which looks like this:

previewNode.SetName(segmentationNode.GetName()+" preview")

mergedImage = vtkSegmentationCore.vtkOrientedImageData()
segmentationNode.GenerateMergedLabelmapForAllSegments(mergedImage,
  vtkSegmentationCore.vtkSegmentation.EXTENT_UNION_OF_EFFECTIVE_SEGMENTS, self.mergedLabelmapGeometryImage, self.selectedSegmentIds)

There was a difference in mergedImage variable:
a) In the situation when I used my contour only, mergedImage is empty
b) If I apply some paint operation, mergedImage contains all the masks (the painted + the automatically segmented)

Questions/Problem solving:

  • What may be wrong/missing in my auto-segmentation, so that “Fill between slices” won’t recognize my masks?
  • To find out more, I tried to debug the “Paint” module. I was not successful. Where in the code can I find the “Paint” module methods?
2

What do you mean by Select contour called “Spinal canal”? You create a segmentation node and create a segment in it (“Spinal canal”)? Does the segment show up correctly in slice views?

Do you mean that slices are filled around the new region only, or throughout the whole image?

Fill between slices only considers visible segments, so make sure you have a display node and the segment is visible. Also, interpolation only happens around slices that have at least one completely empty slice on both sides.

Paint effect is implemented in C++, source code is available on GitHub/Slicer.

3

Thank you for the quick reply!

Yes, exactly. I have a segmentation node, and in it a few segments. One of them and the only selected and visible is “Spinal canal”. It shows up correctly in slice views.

Slices are filled in all the regions between “Spinal canal” segmented slices, i.e. the whole image. This means also in the slices that were not drawn by Paint, but by my owm method.
In effect, I place one point in one slice by Paint and it works the same for all slices (and as intended). But it does not work at all without putting the point via Paint.

The segment is visible (and selected). Slices in-between are empty.

About the display node, I am a bit unsure, whether I have one or not. But I think I have one. I can run
segmentationNode.GetDisplayNode()
and get a vtkMRMLSegmentationDisplayNode as output. Does it mean I do have a display node?

Thanks, so I know where to look and I’ll check it out to see if I can get some hints.

4

If your segment shows up then it means you have a display node.

Do you select a master volume node before you create your automatic segmentation?

How do you create the automatic segmentation? By adding a custom segment editor effect and running it from the Segment editor?

5

Yes. I do select a master volume node, corresponding my loaded data, via

segmentEditorWidget.setMasterVolumeNode(masterVolumeNode)

as in SegmentGrowCutSimple.py

My automatic segmentation method is a separate segment editor effect added to the list of effects (like Paint, Fill between slices or Grow from seeds etc.). It is derived from the SegmentEditorEffect.py example, and modified quite a bit.

It uses numpy and vtk a lot, and convenience functions in slicer.util to convert to segments and back.

6

Does it work correctly if you save the segmentation node to file and then load that file?
If yes, then send me the file and I’ll have a look.

7

Yes, indeed, it works correctly once I load the segmentation from file. I’ve saved the segmentation and I’ll send you the file. Thanks a lot!

8

If everything works if you save and reload the file then maybe you miss a Modified() event. For example, after you’ve finished changing voxels in numpy, you must call arrayFromVOlumeModified() to signal VTK that the changes are ready to be processed.

9

My code for updating the “Spinal Canal” mask reads:

seg2export = vtk.vtkStringArray()
seg2export.InsertNextValue('Spinal Canal')
slicer.modules.segmentations.logic().ExportSegmentsToLabelmapNode(segNode, seg2export, labelmapVolumeNode)
slicer.util.updateVolumeFromArray(labelmapVolumeNode, spinalCanalMask)
slicer.modules.segmentations.logic().ImportLabelmapToSegmentationNode(labelmapVolumeNode, segNode, seg2export)

and within slicer.util.updateVolumeFromArray() the volumeNode.Modified() method is called. There is no modification of the spinalCanalMask beyond these lines of code.

Do you see any problem with this approach in terms of signalling to VTK using Modified(), or any other error?

10

Could you send the link to your complete extension? It does not matter if it is not fully stable or cleaned but, just as it is now.