SegmentEditor consuming tons of memory & 3d slicer crashes

hello,

I have been working to automate some tasks for a set of CT scans which otherwise would take a long time. The code I wrote worked perfectly and in reasonable amount of time until now when I wrote a function where multiple segments (overall four colors) belonging to one category (one color) are merged together using the segment editor and logical operators. This function (if required could post it here) is now the bottleneck with the code not even finishing - I tried couple of things - but in vain - I also optimized my code where possible I thought I could - reducing time by 10-12 seconds, but still it takes a long time -and eventually 3d slicer crashes. I tried this on both - Mac (8 gigs RAM) and Windows (16 gigs RAM).

To figure out the bottleneck in more detail, I profiled my code and the attached snapshot shows the total time taken. I am a bit confused as to how this merging process can take so much time programmatically although when I do manually it is smooth and super fast. As can be seen from the profile report/analysis, the SegmentEditorThresholdEffect and SegmentEditorLogicalEffect are the two main functions that take most of the time. I have disabled volume rendering, display etc. but not much effect.

The function does this: There is a segmentation comprising of multiple RGBY segments and I have to merge them into just these four colors. So i add a new segment to my segmentation (empty) and then perform logical operations (union) on each segment of a particular color (modifier segment) and this new target segment and then remove the individual ones.

Can someone please share some insight as to what I could be doing wrong or some advice/tip? I can gladly share the code if required.

Edit 1: I added the code, should give more idea as to what I am doing. Let me know please if some info is required

CODE:

# segment editor part
def combine_same_colors(folder_num, bone_obj_seg_list, final_segmentation_node, final_segmentation):
	# naming and color vals
	vol_name = folder_num + 'HRimage'
	color_dict = {'blue': (0, 0, 1), 
				  'red': (1, 0, 0), 
				  'green': (0, 1, 0), 
				  'yellow': (1, 1, 0)}

# set up the segment editor widget and node
	vol_node = slicer.util.getNode(vol_name)
	seg_edit_wid = slicer.qMRMLSegmentEditorWidget()
	seg_edit_wid.setMRMLScene(slicer.mrmlScene)
	seg_edit_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSegmentEditorNode")
	seg_edit_wid.setMRMLSegmentEditorNode(seg_edit_node)
	seg_edit_wid.setSegmentationNode(final_segmentation_node)
	seg_edit_wid.setMasterVolumeNode(vol_node)

	seg_edit_wid.setActiveEffectByName("Logical operators")
	effect = seg_edit_wid.activeEffect()
	effect.setParameter("Operation", "UNION")
	segments_list = [i[0].get_segment() for i in bone_obj_seg_list]
	colors = ['blue', 'red', 'green', 'yellow']
# code to check if there are multiple segments belonging to same color and if yes, merge them
	def func(color):
	    	return lambda x: True if color in x else False
	for i in colors:
		check_multiple_seg = Counter(map(func(i), segments_list))[True]
		if check_multiple_seg > 1:
			color_seg_list = [j for j in segments_list if i in j]

			segment_name = folder_num + 'lego' + i
			final_segmentation.AddEmptySegment(segment_name)
			final_segmentation.GetSegment(segment_name).SetColor(color_dict[i][0],
																 color_dict[i][1],
																 color_dict[i][2])

			# target_seg_id = final_segmentation_node.GetSegmentation().GetSegmentIdBySegmentName(segment_name)
			target_seg_id = final_segmentation.GetSegmentIdBySegmentName(segment_name)
			# source_seg_id = [final_segmentation_node.GetSegmentation().GetSegmentIdBySegmentName(i) for i in color_seg_list]
			source_seg_id = [final_segmentation.GetSegmentIdBySegmentName(i) for i in color_seg_list]

			seg_edit_node.SetSelectedSegmentID(target_seg_id)

			for i in source_seg_id:
				effect.setParameter("ModifierSegmentID", i)
				effect.self().onApply()
				final_segmentation.RemoveSegment(i)

	return final_segmentation_node, final_segmentation

Thank you very much.

Screenshot 2021-08-01 at 3.42.50 PM

Windows Errors: (I also increased the virtual memory in Win 10, but no improvement)

err1

As a hint to help you get help on this, it’s good to provide a way for others to replicate the issue easily (replicate on sample data, provide a one-line way to replicate the issue, etc). These guidelines can help: http://sscce.org/

@pieper hello, thanks for the reply. Right, let me refactor it so that it could be run on on other machines as well. On it!

Meanwhile if the above info can provide some clues, would be nice! :slight_smile:

Combining segments is a very complex operation, because it has to take into account all 3 kinds of masking methods and needs to manage multiple segmentation layers.

If your segments are not overlapping then it should be simpler and faster to combine the segments while they are still stored in a labelmap (in a numpy array) and import the result into a segmentation node.

@lassoan Thank you for the advice. I have three questions with respect to this:

  1. I am a bit uncertain as to how do you combine segments while they are in a labelmap volumes, like you mentioned. I have my ROIs as STL files which I load segmentations in 3d slicer and then export them to binary label map. And now when I have these label maps of different segmentations, how to get them together?

Right, so on experimenting I found two modules that go by the name - “Add scalar volume” and “Image Label Combine”. The first one is, from what I understand for scalar volumes whereas the second one is specifically for combining labelmap volumes of same sizes. Am I correct here or is there some other piece of functionality in 3D slicer that exists and that I am missing here? The problem I get with this is - I combined labelmap volumes using the Image Label Combine module, however, in the final combined label map volume, the segmentations lose their color and they become all of one color which is not what is desired. This is surprising because in the segmentations module and in their own individual label map volumes, they still retain their color.

  1. This one I just thought of as I remembered that my original ROIs are in STL format. So I exported them to models and then using the “Merge Models” module, combined them and then exported to segmentation node. It worked - what I would be glad to know that if I automate this then i hope this is not a complex operation like with segment editor where manually it worked fine but on automating, it blew up

  2. After researching, I am not able to find much documentation code related to the two modules above - is it possible to do the above through Python (merge models and image label combine modules)?

P.S - @lassoan Could you please guide as to how this complex procedure could be avoided and done without segment editor. I found an old post where you mentioned that segmenteditor is the way to go ahead (Merging two models) but as can be seen, it might not be the best way - so if you could expand a bit on your previous suggestion - i would really appreciate it.

Thank you.

Conversion from closed surface to labelmap is best done by Segmentations module.

Combining non-overlapping segments is internally complex (due to the reasons I described above, it also consumes more memory due to saving undo information, etc). If all you need is to change labels in a simple 3D volume then even “Image Label Combine” module is an overkill nowadays. You can simply export the segmentation to labelmap volume and manipulate its voxels as a numpy array, using standard numpy operations. For example, if you get the voxels as a numpy array a then you can merge all non-zero labels (set them to 1) by calling: a[a>0]=1.