# Segment Editor Effect: Fill Between Slices does not recognize existing contour

**URL:** <https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452>\
**Category:** Development\
**Tags:** segmentation\
**Created:** [July 10, 2018, 9:01pm UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452 "2018-07-10T21:01:14Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![MRXCAT\_CMR](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@MRXCAT\_CMR](https://discourse.slicer.org/u/MRXCAT_CMR)\
**Post date:** [July 10, 2018, 9:01pm UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/1 "2018-07-10T21:01:14Z")

</div>

Operating system: Windows 10  
Slicer version: 4.8.1 (r26813) built myself

**Background:**  
I have started some work on automatic segmentation using the Segment Editor framework. Now I have an automatically generated contour for the spinal canal. The contour is present in some slices (about every 3rd), and is not present in others. Now I thought about including some methodology from “Fill between slices” module. The first attempt before automating is testing manually by clicking through the GUI.

**Approach - step by step**

1. Select contour called “Spinal canal” and open “Fill between slices” module
2. Click “Initialize”
3. Check the result by using the slider - the results contour is empty
4. Then I selected the “Paint” module for some testing, draw a small region in an empty slice of “Spinal canal” contour.
5. Now try “Initialize” again in the “Fill between slices” module, and wow, the result looks good.

**Problem analysis**  
Obviously, there’s something wrong which makes my automatically segmented structure not being recognized by the “Fill between slices” module. But what?

I analyzed the source and had a look at the file `AbstractScriptedSegmentEditorAutoCompleteEffect.py`, and the `preview` method, there around line 342, which looks like this:

```
previewNode.SetName(segmentationNode.GetName()+" preview")

mergedImage = vtkSegmentationCore.vtkOrientedImageData()
segmentationNode.GenerateMergedLabelmapForAllSegments(mergedImage,
  vtkSegmentationCore.vtkSegmentation.EXTENT_UNION_OF_EFFECTIVE_SEGMENTS, self.mergedLabelmapGeometryImage, self.selectedSegmentIds)

```

There was a difference in `mergedImage` variable:  
a) In the situation when I used my contour only, `mergedImage` is empty  
b) If I apply some paint operation, `mergedImage` contains all the masks (the painted + the automatically segmented)

**Questions/Problem solving** :

- What may be wrong/missing in my auto-segmentation, so that “Fill between slices” won’t recognize my masks?
- To find out more, I tried to debug the “Paint” module. I was not successful. Where in the code can I find the “Paint” module methods?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 11, 2018, 12:21am UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/2 "2018-07-11T00:21:00Z")

</div>

What do you mean by _Select contour called “Spinal canal”_? You create a segmentation node and create a segment in it (“Spinal canal”)? Does the segment show up correctly in slice views?

> [@MRXCAT\_CMR](#):
>
> “Initialize” again in the “Fill between slices” module, and wow, the result looks good.

Do you mean that slices are filled around the new region only, or throughout the whole image?

> [@MRXCAT\_CMR](#):
>
> What may be wrong/missing in my auto-segmentation, so that “Fill between slices” won’t recognize my masks?

Fill between slices only considers visible segments, so make sure you have a display node and the segment is visible. Also, interpolation only happens around slices that have at least one completely empty slice on both sides.

> [@MRXCAT\_CMR](#):
>
> To find out more, I tried to debug the “Paint” module. I was not successful. Where in the code can I find the “Paint” module methods

Paint effect is implemented in C++, source code is available on [GitHub/Slicer](https://github.com/Slicer/Slicer/blob/master/Modules/Loadable/Segmentations/EditorEffects/qSlicerSegmentEditorPaintEffect.cxx).

---

<div class="post-metadata">

**Author:** ![MRXCAT\_CMR](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@MRXCAT\_CMR](https://discourse.slicer.org/u/MRXCAT_CMR)\
**Post date:** [July 12, 2018, 8:53am UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/3 "2018-07-12T08:53:53Z")

</div>

Thank you for the quick reply!

> [@"lassoan](#):
>
> What do you mean by _Select contour called “Spinal canal”_? You create a segmentation node and create a segment in it (“Spinal canal”)? Does the segment show up correctly in slice views?

Yes, exactly. I have a segmentation node, and in it a few segments. One of them and the only selected and visible is _“Spinal canal”_. It shows up correctly in slice views.

> [@"lassoan](#):
>
> > [@MRXCAT\_CMR](#):
> >
> > “Initialize” again in the “Fill between slices” module, and wow, the result looks good.
> 
> Do you mean that slices are filled around the new region only, or throughout the whole image?

Slices are filled in all the regions between “Spinal canal” segmented slices, i.e. the whole image. This means also in the slices that were not drawn by Paint, but by my owm method.  
In effect, I place one point in one slice by Paint and it works the same for all slices (and as intended). But it does not work at all without putting the point via Paint.

> [@"lassoan](#):
>
> > [@MRXCAT\_CMR](#):
> >
> > What may be wrong/missing in my auto-segmentation, so that “Fill between slices” won’t recognize my masks?
> 
> Fill between slices only considers visible segments, so make sure you have a display node and the segment is visible. Also, interpolation only happens around slices that have at least one completely empty slice on both sides.

The segment is visible (and selected). Slices in-between are empty.

About the display node, I am a bit unsure, whether I have one or not. But I think I have one. I can run  
`segmentationNode.GetDisplayNode()`  
and get a `vtkMRMLSegmentationDisplayNode` as output. Does it mean I do have a display node?

> [@"lassoan](#):
>
> > [@MRXCAT\_CMR](#):
> >
> > To find out more, I tried to debug the “Paint” module. I was not successful. Where in the code can I find the “Paint” module methods
> 
> Paint effect is implemented in C++, source code is available on [GitHub/Slicer](https://github.com/Slicer/Slicer/blob/master/Modules/Loadable/Segmentations/EditorEffects/qSlicerSegmentEditorPaintEffect.cxx).

Thanks, so I know where to look and I’ll check it out to see if I can get some hints.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 12, 2018, 12:55pm UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/4 "2018-07-12T12:55:09Z")

</div>

If your segment shows up then it means you have a display node.

Do you select a master volume node before you create your automatic segmentation?

How do you create the automatic segmentation? By adding a custom segment editor effect and running it from the Segment editor?

---

<div class="post-metadata">

**Author:** ![MRXCAT\_CMR](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@MRXCAT\_CMR](https://discourse.slicer.org/u/MRXCAT_CMR)\
**Post date:** [July 12, 2018, 8:30pm UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/5 "2018-07-12T20:30:18Z")

</div>

> [@lassoan](#):
>
> If your segment shows up then it means you have a display node.
> 
> Do you select a master volume node before you create your automatic segmentation?
> 
> How do you create the automatic segmentation? By adding a custom segment editor effect and running it from the Segment editor?

Yes. I do select a master volume node, corresponding my loaded data, via

`segmentEditorWidget.setMasterVolumeNode(masterVolumeNode)`

as in [SegmentGrowCutSimple.py](https://gist.github.com/lassoan/2d5a5b73645f65a5eb6f8d5f97abf31b)

My automatic segmentation method is a separate segment editor effect added to the list of effects (like Paint, Fill between slices or Grow from seeds etc.). It is derived from the [SegmentEditorEffect.py](https://github.com/Slicer/Slicer/blob/master/Extensions/Testing/ScriptedSegmentEditorEffectExtensionTemplate/ScriptedSegmentEditorEffectModuleTemplate/SegmentEditorScriptedSegmentEditorEffectModuleTemplateLib/SegmentEditorEffect.py) example, and modified quite a bit.

It uses numpy and vtk a lot, and convenience functions in `slicer.util` to convert to segments and back.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 13, 2018, 3:02am UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/6 "2018-07-13T03:02:33Z")

</div>

Does it work correctly if you save the segmentation node to file and then load that file?  
If yes, then send me the file and I’ll have a look.

---

<div class="post-metadata">

**Author:** ![MRXCAT\_CMR](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@MRXCAT\_CMR](https://discourse.slicer.org/u/MRXCAT_CMR)\
**Post date:** [July 16, 2018, 10:35am UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/7 "2018-07-16T10:35:59Z")

</div>

> [@Segment Editor Effect: Fill Between Slices does not recognize existing contour](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/6):
>
> Does it work correctly if you save the segmentation node to file and then load that file? If yes, then send me the file and I’ll have a look.

Yes, indeed, it works correctly once I load the segmentation from file. I’ve saved the segmentation and I’ll send you the file. Thanks a lot!

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 17, 2018, 4:23am UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/8 "2018-07-17T04:23:19Z")

</div>

If everything works if you save and reload the file then maybe you miss a Modified() event. For example, after you’ve finished changing voxels in numpy, you must call [arrayFromVOlumeModified()](https://github.com/Slicer/Slicer/blob/master/Base/Python/slicer/util.py#L760-L769) to signal VTK that the changes are ready to be processed.

---

<div class="post-metadata">

**Author:** ![MRXCAT\_CMR](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@MRXCAT\_CMR](https://discourse.slicer.org/u/MRXCAT_CMR)\
**Post date:** [July 17, 2018, 1:06pm UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/9 "2018-07-17T13:06:06Z")

</div>

> [@lassoan](#):
>
> If everything works if you save and reload the file then maybe you miss a Modified() event. For example, after you’ve finished changing voxels in numpy, you must call [arrayFromVOlumeModified()](https://github.com/Slicer/Slicer/blob/master/Base/Python/slicer/util.py#L760-L769) to signal VTK that the changes are ready to be processed.

My code for updating the _“Spinal Canal”_ mask reads:

```
seg2export = vtk.vtkStringArray()
seg2export.InsertNextValue('Spinal Canal')
slicer.modules.segmentations.logic().ExportSegmentsToLabelmapNode(segNode, seg2export, labelmapVolumeNode)
slicer.util.updateVolumeFromArray(labelmapVolumeNode, spinalCanalMask)
slicer.modules.segmentations.logic().ImportLabelmapToSegmentationNode(labelmapVolumeNode, segNode, seg2export)

```

and within `slicer.util.updateVolumeFromArray()` the `volumeNode.Modified()` method is called. There is no modification of the `spinalCanalMask` beyond these lines of code.

Do you see any problem with this approach in terms of signalling to VTK using `Modified()`, or any other error?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 17, 2018, 3:24pm UTC](https://discourse.slicer.org/t/segment-editor-effect-fill-between-slices-does-not-recognize-existing-contour/3452/10 "2018-07-17T15:24:17Z")

</div>

Could you send the link to your complete extension? It does not matter if it is not fully stable or cleaned but, just as it is now.