Add Segment to SegmentationNode uses invalid color values (scalar) and adds a new colortable node

Support
, ,
1

Summary:

I am attempting to import/edit/export segmentations from numpy to slicer to numpy. I am using custom color tables but am getting invalid color IDs on export after a segment has been added.

For example, my color table file defines 118 colors from IDs 0 to 117. I load a segmentation, set the color node to my loaded color file, then when I add a new segment in Segment Editor to that segmentation node, I select the appropriate color for it from the loaded terminology. However, on export the newly added segment has segment ID 118, not the correct value that would be found on lookup in either the terminology JSON file or the color table node. I noticed that in the All nodes tab of slicer a new color table node now exists with name [segmentation node name]_ColorTable and that new colortablenode contains an extra segment with ID 118 and the name from the terminology file. So rather than using the color table node that I’ve defined it’s just making one up. I’m using the code found here - Script repository — 3D Slicer documentation - for importing/exporting but it doesn’t seem to be working as I would expect? Is there some extra python code that I need to include to ensure that segments added to an existing segmentationnode use that node’s assigned colortablenode for name/id lookups?

2

Here is some code:

Within my slicer extension Widget class I load a custom terminology file:

path = self.resourcePath("Terminologies/[terminology filename].json")
tlogic = slicer.modules.terminologies.logic()
tlogic.LoadTerminologyFromFile(path)

Within my slicer extension module’s [Module]Lib folder [name].py file:

This is the code that loads a numpy array into a labelmapvolume, assigns the custom color table, and then imports the labelmapvolume node into a segmentationnode using a custom terminology file (which matches the custom color table).

# load numpy array into slicer node: 
vol, affine = getvol(...) # loads numpy arrays for labels and affine transform

name = 'myNumpyImportNode'
nodetype = "vtkMRMLLabelMapVolumeNode"
arraytype = vtk.VTK_INT

node = slicer.mrmlScene.AddNewNodeByClass(nodetype, name)

vtkArray = vtk.util.numpy_support.numpy_to_vtk(
    num_array=vol.ravel('F'), deep=True, array_type=arraytype
)
image = vtk.vtkImageData()
image.SetDimensions(vol.shape)
image.GetPointData().SetScalars(vtkArray)
# Set the volume node's "image" data
node.SetAndObserveImageData(image)
transformMatrix = vtk.vtkMatrix4x4()
for i in range(4):
    for j in range(4):
        transformMatrix.SetElement(i, j, affine[i, j])
node.SetIJKToRASMatrix(transformMatrix)        

color_table_node = slicer.util.loadColorTable([color_file.txt])
color_table_node.SetNumberOfColors(num_colors)

node.CreateDefaultDisplayNodes()
node.GetDisplayNode().SetAndObserveColorNodeID(color_table_node.GetID())

# import to segmentation node and apply terminology (which matches color table node in terms of Segment ID:Name lookups) 
seg_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSegmentationNode")
slicer.modules.segmentations.logic().ImportLabelmapToSegmentationNodeWithTerminology(node, seg_node, [terminology name])

Is it bad practice to import the labelmap with terminology here since a colortable has already been defined? The colortable and terminology files match in terms of id:segment name lookups.

<Edit the segmentation node using SegmentEditor, add a segment, assign segment name by double-clicking on the new segment color box to bring up the Terminology selector, select the correct terminology for the new segment, close. assign some pixels to that segment.>

This is the code that saves the edited segmentation node back to numpy:

seg_node = slicer.util.getNode([name of edited segmentation node])
# lbl_node.GetClassName() == 'vtkMRMLSegmentationNode'
vol_name = lbl_node.GetName()
tmp_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLLabelMapVolumeNode")
<re-load the original, unedited numpy array into a reference node> 
ref_node = slicer.util.getNode([name of unedited labelmapvolume node])

ctb_node = seg_node.GetDisplayNode().GetColorNode()
segmentIds = seg_node.GetSegmentation().GetSegmentIDs()
slicer.modules.segmentations.logic().ExportSegmentsToLabelmapNode(seg_node, segmentIds, tmp_node, ref_node, slicer.vtkSegmentation.EXTENT_REFERENCE_GEOMETRY, ctb_node)
# ExportSegmentsToLabelmapNode() adds a new (unnecessary) color table node to the list of nodes with name [name of edited segmentation node]_ColorTable. This colortablenode contains an incorrect lookup value for the new class. 

tmp_node.GetDisplayNode().GetColorNode().GetName() # the color table node assigned to the exported labelmapvolume node is this new color table node not the one passed to the export function. 

# convert to num_py arrays
tmp_array = slicer.util.arrayFromVolume(tmp_node)
ref_array = slicer.util.arrayFromVolume(ref_node)

# check the contents: 
tmp_vals, tmp_cnts = np.unique(tmp_array,return_counts=True)
ref_vals, ref_cnts = np.unique(ref_array,return_counts=True)

It is unclear why an additional ColorTableNode is created by ExportSegmentsToLabelmapNode(). This new (seemingly unnecessary) color table node is added to the list of nodes in the “All nodes” tab of the Data module with name [name of edited segmentation node]_ColorTable.

tmp_vals contains one additional segment ID value for the added segment. Ideally this value should match the ID that is found for that segment name/color record in the terminology/color table file. i.e., if i added ‘blah’ segment name corresponding to ID = 8 in the color file then the new ID added should be 8. Sometimes the new ID is 8 and sometimes it is assigned the value n+1 where where n is the max ID found in the color table file. Based on this function: https://github.com/Slicer/Slicer/blob/294f0baf9ad23ed1e7ebf96f2ff3bb15fda12f3c/Modules/Loadable/Segmentations/Logic/vtkSlicerSegmentationsModuleLogic.cxx#L2494 This seems to be the expected behavior when a segment name is not found on lookup. … and now that I’ve tested this theory it is what is causing the problem. In the Terminology JSON file and the Color Table files, all of my segment names that contain underscores are not found on lookup whereas the ones without underscores are found. So it seems that the functions vtkSlicerTerminologiesModuleLogic::GetColorIndexByTerminology() and GetColorIndexByName(segmentName) will not successfully find segment IDs if the segment names contain underscores in the color table or terminology file. If so, then this seems like a bug.

I’m using the original colortable file formatting (.txt file) where underscores in the segment names are required due to the use of spaces as separators. I believe that these underscores are getting incorrectly stripped out during the segment name:ID lookup because (a) the segmentation node segment names DO NOT contain underscores, (b) the Terminology modal for selecting a segment name for the new segment DOES contain underscores, (c) the segmentation node segment name for the newly added segment DOES contain underscores if they were there in the first place, (d) the color table file and terminology files both contain underscores for some segment names, (e) only the segment names containing underscores fail the lookup.

I can think of a few work-arounds for this:

  1. switch to the new CSV format of color table file which allows spaces in the segment names
  2. update my terminology JSON file to replace underscores with spaces as this file has no restriction on the use of spaces in segment names.

but I suspect you want the segment name to ID lookup to work even if the segment names contain underscores in the source terminology and/or color table files?

3

I can confirm that updating the terminology JSON file to replace underscores with spaces does seem to fix the failed lookup.

4

Please use the new standard .csv file based format. The old custom txt format used space as separator between columns, so space in color name was replaced by underscores, so underscores were converted to spaces. You don’t have this limitation if you use the new format.

I would also recommend to use standard terminology in the color table file. If terminology is set then matching will be done based on standard codes. It is much more reliable than a simple string comparison, where spelling, capitalization, typos, etc. differences (which are particularly common when you aggregate data from multiple sites or from multiple operators) may prevent matching of segments.

ExportSegmentsToLabelmapNode always creates a new color table because you need a way to verify if there were any segments that were not present in the exportColorTable that you provided. After you completed all your checks that you wanted to do, you can remove the color table by a single slicer.mrmlScene.RemoveNode call. If this API is inconvenient then we could add a flag or a new variant of ExportSegmentsToLabelmapNode, which would require the exportColorTable and would return with failure if a segment was not found in the exportColorTable. This varian would not need to create a new color table node, it would just assign exportColorTable to the exported labelmap node.