Hi, I’m currently exploring the use of nnInteractive slicer’s version and noticed that the segmentation cuts pretty steaply not finishing the segmentation of the tibia or femur for example. Maybe Andras or Steve could tell me if there’s a number of slicer or voxel limitation and if it’s possible to overpass this limitation so I can segment my full bones.
On a side note, for people having difficulties installing and starting the server for the extension on Windows, I found a work around until the main GitHub founds a more optimize solution, so let me know if you need my instructions on the subject.
For exemple with image representing a whole femur. When prompt the head, it loads the orange section. If asked lower, it’s the green section and stopping at the mid point of the scrobbly section. When prompt a second point on the same segmentation, it joins the segments with this ugly and none accurate finish.
Like any user guided tool, NNinteractive will not give you a perfect segmentation in the first attempt. But there are no inherent limits to it (beyond your GPU size, if you are using the GPU version). You need to “guide” the segmentation by providing positive (parts belong) and negative (parts don’t belong) to the segment. You also chose to segment head as a different segment, so it kind of makes sense for it not to extend it below the other segment.
I suggest merging these two segments, and then adding positive scribble to that missing section, and negative scribbles to the erroneous part in the middle.
I chose to do a first segmentation at the head to show that it stops and don’t do the full bone.
Green parts are to show the effect of two consecutive segmentation and the finish that it gives. The example on the GitHub shows longer segmentation with the tool and I tried with a RTX 2080 TI, 5070 super and 5090 and same length, so no effect on the vram or GPu limitation.
Yes, that’s exactly how you should use it. You need to guide the segmentation. It doesn’t know what a femur (or any other structure) looks like. It just knows things like edges, boundaries, curves. And you need to help it.
Yes, but still limiting to diaphyse part of the bone. Almost exactly the same slice each time. My main question was to know if there was a voxel limitation. Steve answered.
nnInteractive has an auto-zoom feature that should help with getting a complete segmentation of a structure most of the time. From the nnInteractive paper:
nnInteractive dynamically expands the region of interest (ROI) based on prediction borders, iteratively zooming out by a factor of 1.5 until the object is fully captured (up to 4× zoom out)
It is a heuristic method, so it is not guaranteed to always work perfectly, and whenever encounter oversegmentation or undersegmentation then you can just add additional hints to include less or more.
However, it could be possible to slightly adjust the auto-zoom method to work well for your case. Your case is a bit unusual in that the bone is very long and it is only a thin shell. At the zoom level where the complete bone would be visible in one patch, the cortical bone might become less than a single voxel thick or it may not separable from other bones. It could be a good test case for the auto-zoom method, so I would suggest to submit it to the nnInteractive project issue tracker.
Also, I noticed that from the results I get with TotalSegmentator, nnInteractive has a smoother finish as if it was applying a smoothing mask after the segmentation. I know TotalSegmentator use a binary method of including a voxel or not in the segment, but I’m not sure of the way nnInteractive gets to this smoothness from the first try.
Here’s the result with TotalSegmentator just to show the difference of quality.
