Background: we are writing custom application with web UI for segmentation of 2D+time data. The semi-automatic algorithm relying on grow from seeds works well, but not all the time. So manual corrections are needed. Ideally, convert the label map into a 2D spline for each time point, to allow easy interaction. The structure of interest is smooth in these images, so smoothing of splines is a welcome side-effect. Converting that into a fractional label map is preferred over binary label map.
Segmentation module’s documentation mentions these conversions. It was already answered that direct editing of contours is not possible with Slicer. Is this still true? How hard would that be with an API, if one exists? Reuse is preferred over reimplementation.
Is conversion from binary label map into planar contours best done by converting to 3D mesh first, then cutting the mesh? Is that already implemented in Slicer, and where?
Is there an easy or already implemented way of converting the resulting 2D polygon into a spline? Pointers into relevant code are appreciated, as vtkLinearSpline.cxx does not contain a lot of code.
The last question about this was two years ago. Did anything important change? Can someone point me to relevant pieces of code for ripping it out of Slicer and into a stand-alone executable?
Storing and managing segmentation data in a series of 2D contours is a legacy method, and so far we only needed it for DICOM-RT Structure Set, and did not want to encourage its use (especially that there is the DICOM Segmentation Object, which is much superior to RTSS). This is why we did not even implement a conversion rule going from labelmap to planar contour, but the code that does the conversion is in the exporter function itself:
With the recent addition of the new Markup type of closed curve this would be much easier to do than before. The only thing we’d need is a converter from/to planar contour segmentations (and the abovementioned labelmap → contours conversion, since the other way is implemented already).
I agree that current markups curves should be able to handle editing quite well.
We just need an algorithm for getting closed curves from segmentation/slice intersections. It would require two steps:
segmentation slicing (cut segmentation with slice): The implementation in the segmentation 2D displayable manager and in the exporter (that @cpinter linked above) are about the same and should be usable for simple shapes. For complex cases you would need to use vtkContourTriangulator class but we could not switch to it because it has a small but serious bug that may cause the application hang for invalid inputs (see details here: https://github.com/Slicer/SlicerGitSVNArchive/pull/1075).
spline fitting: we could create a spline by resampling the linearly interpolated contour at uniform distances and use these as control points
Hello, I was wondering if there have been any updates on this? @lassoan@cpinter I would be interested in fitting splines (perhaps using the closed curve markup) and manually refining the segment in the 2D views so that it is reflected smoothly in the 3D segmentation. Is there an existing module that offers this functionality? Any assistance would be greatly appreciated.
vtkContourTriangulator now works robustly (does not hang anymore), so it could be used for slicing the segmentation
We have now curve-based segmentation editing in mulitple effects, which could be used as an example. Draw tube and Surface cut effects in SegmentEditorExtraEffects extension use curves/markups to edit segmentation.
Therefore, it would now be easier to implement editing segmentations using 2D contours.
Can you write a bit about your use case? I’m just wondering whether the need for spline-based segmentation editing is still there, now that so many “AI” segmentation tools are available. It may be possible that the need is even stronger, because we want to be able to quickly fix up automatic segmentation results. But then maybe what we would really need is 3D surface editing, and not tedious slice-by-slice editing. Surface cut effect does something like this, but it it is quite limited (does not work well for concave shapes; cannot initialize the surface from an existing segmentation).
Reflecting on this “3D surface editing” topic. I developed something like this almost 20 years ago in MITK. Posting part of a screenshot I have from that time:
The idea was that you “grow” a ROI on a triangle mesh, starting from a point and including cells as you keep it growing. Then “pull out” or “push in” this patch using a function (in the screenshot Gaussian is selected, but if I remember correctly the alternatives were Sine and Triangle). The goal was manual fixing of automated segmentation results.
It worked nicely, but an awfully complicated surface regularization post-processing step was needed due to the malformed triangles resulting after an operation, and due to possible self-intersections etc.
However, the project was dropped because, at least at that time, it was not considered to be easy enough to use. I was not involved in the testing phase, but you @lassoan may have been, as this was part of a collaboration with GE Healthcare and you were one of our contacts at the company back then.
Yes, what I am looking for is a tool to manually refine automatic segmentations or even for manual labeling. Yes, I mean 3D surface editing, but I think it would be easier to make corrections by “pulling out” or “pushing in” directly from the slice view, while ensuring the corrections are applied to the 3D segmentation.
In theory something like this could be developed in a way that operates on labelmaps in the end, which is good because 1) that’s the source representation and 2) there won’t be mesh integrity issues. Unfortunately a project like this is not something small, I’d say at the very least a master’s project.
