We are working with CT scan source images to segment bone. High resolution segments are needed. Each source Cropped Volume is around 1400x600x300 px. Grow from seeds is our actual approach.
Actual configuration:
Operating system: Win10
Slicer version: 4.11.0-2019-11-15 r28624
Graphic card: Nvidia GTX 1060 6G
RAM: 48Gb DDR3 1333
2x Xeon E5 2670
SATA 3 SSD 240Gb
Each segmentation made with Grow from seeds takes around 10 min. Any paint tast to improve/fix segments takes around a minute so autoupdate is not very usable. Other edit segment tools are affected. Margin Tool - Grow is a full night task, for example.
Checking system monitor, we saw some interesting items:
Just one core is 100%, the others are most 0%
RAM is almost full
Main thread is almost blocked.
Questions:
How can we improve/change hardware for ideal configuration?
Have CUDA drivers any role in this task? Which is best Nvidia Driver?
Any application setting change could improve computation times?
RAM almost full is probably the main issue. Could you please try how the performance changes if you slightly crop/downscale the image before segmentation using Crop volume module?
Grow from seeds builds a graph representation of the image, which currently requires about 80 bytes per voxel, that’s why the memory usage jumps up so much. We could probably reduce it by about 50% without significant change in the algorithm. If indeed memory usage is the main culprit then this could help a lot.
If you use such a huge kernel for margin then it means that you lose all the small details in the image. You might get equivalent results by downsampling the input by a factor of 5 along each axis (reducing memory need by a factor of 100x). Try Crop volume module with setting spacing scale to 5.0 to create a downsampled image and segment that.
Ok. I will do a new segmentation from a less resolution source volumen, aply margin and send the segmented item to the high resolution segmentation… It´s a bit tricky but i think it should work. Is there other way to make a low resolution growth from a segment?
On the other hand, recovering initial question about Grow from seeds. How can I tried this “non significant change in the algorithm”?
Is margin effect a final step in your workflow? If it is then it means there there will be no fine details in your segmentation results, so you may just as well do the complete segmentation workflow on a lower-resolution volume.
What kind of data you work with? What do you need to segment? Why do you need the huge margin?
We could change indices to use 32-bit indexes instead of the current 64-bit indexes and 64-bit pointers. That could reduce memory usage by about about a factor of 2x. However, if you can increase spacing scale by 2.0 (and still get appropriate results) then memory usage would drop by a factor of 8x (and scale of 5.0 would result in 125x less memory used).
Margin Effect:
I need a high resolution source volumen because I want high accuracy bone segmentations. I need to grow some segments to make logical operations and combine them with the others. This is the reason not to downsample segmentation.
Grow from Seeds:
Reason is the same. If I drop/copy a low resolution segment to the high resolution segmentation, surface is not smooth (stepped surface)
Can you post a few screenshots so that we can have an idea about the kind of images you work with? Is the spacing of the image isotropic? Have you tried WrapSolidify effect to fix discontinuities?
I make a new low resolution segmentation and copy segment to it. I apply gaussian smooting and margin grow 6mm. “Show 3d Surface Smoothing” is ON because “you send what you see”.
“Show 3d Surface Smoothing” is ON in the high resolution segmentation too.
I don´t know why is so good looking at begining but it decrease quality when you make any change on it.
When you copy a high-resolution segment to a low-resolution segmentation then the segment is not converted immediately, only when needed (e.g., when the segment is edited). If the segmentation’s resolution is isotropic (spacing is the same along all axes) then surface smoothing can remove step artifacts without losing details.
Both source volumes are made in isotropic way from begining.
Low resolution segmentation is making good job with sended segment as you see in images. The real problem is from low to high resolution segmentation.
I have tried to export smoothed low resolution segment as stl and import it again to high resolution segmentation and quality is as perfect as expected (but i have to turn 180Âş model with a transform). Tricky workflow but effective.
Why do you need to export and reimport to file?
If you export in RAS coordinate system then there is no need for LPS->RAS conversion after import (180deg rotation).
@lassoan your comments regarding cropping and downsampling for the current robust solution are all spot on. If anyone wants to devote the engineering time for a faster solution, you might want to explore the Felzenszwalb and Huttenlocher method for computing Euclidean Distance Transforms. This is elegantly described by Philip Rideout who provides elegant Python code. This method is far faster than computing erosion, providing a separable filter for each of dimension. For each dimension, you can compute the rows in parallel, leveraging multiple CPUs.
I provide a sample project Thick3D that demonstrates parallel acceleration and can work with NIfTI and NRRD input. Since the code handles parallel threading, anisotropic images and other situations, it is not as elegant as Rideout’s code for porting into Slicer3D, but it fully demonstrates the potential for solving these problems with tremendous speed and with very little RAM.
Its not a one-to-one replacement for the current algorithm - it is really designed for binary images. However, it might provide a real-time method for some situations.
This code is ready to be stolen. As Maurer, Not and Threshold filters are parallelized, this should scale with number of CPU cores, which is not the case with regular morphology. Not to mention that the size of radius mostly doesn’t matter for computation time.
I’ve updated Grow from seeds effect with these changes and did some performance optimizations. Now peak memory usage is about half than before and the effect is about 20% faster.
@Chris_Rorden@dzenanz Thanks for the suggestions. We haven’t thought about using distance transforms for margin and hollow effect, but of course it makes complete sense.
@Chris_Rorden do you know if Felzenswalb&Huttenlocher or Rideout methods have any advantage over ITK’s distance map filters? Felzenswalb&Huttenlocher writes this about Maurer’s method (one of the methods implemented in ITK):
There are a number of other algorithms for computing the EDT of a binary image in linear-time (e. g., [18, 7, 17]); however these methods are quite involved and are not widely used in practice. In contrast, our algorithm is relatively simple, easy to implement, and very fast in practice.
This is not just vague but even misleading, since for example ITK’s signed Maurer distance is used very widely. Considering obvious bias of the authors and absence of qualitative or quantitative comparison to established methods, I don’t feel much motivation to spend time with evaluating their method, but if @Chris_Rorden or others have first-hand experience then it would be great to learn about it.
@Sunderlandkyl Could you look into using distance map images instead of vtkImageDilateErode3D in Segment editor? Not urgent, just when you can get to it. Simple distance map computation + thresholding (as shown in @dzenanz example) should work. Since we don’t rely much on SimpleITK in Slicer core modules, it would be probably better not to add a dependency now, but instead add a new “vtkITKImageMargin” filter, which would offer erosion, dilation, and hollowing by wrapping the ITK filters. If morphological filter performance is better for small kernels then we might keep both methods and use morphological operators for small kernel sizes and switch to distance map for large kernels.
@lassoan I suspect the Maurer method performs similarly to F&H. Several groups seem to have independently discovered separable ways to compute distance fields. If you are interested in the different methods, the history of their discovery, as well as clever optimizations, I strongly recommend William Silversmith’s Github project. That page also notes a trick for accelerating the estimate for the first dimension using a simple forward and back sweep. Finally, Silversmith also describes how these algorithms can be modified to handle anisotropic images. This is important, as anisotropic images are typically not handled by erosion methods or these separable filters.
So while there are variations on these methods, and a few tricks to accelerate any implementation, I think all are much faster and have lower memory requirements than traditional methods. If the Maurer method is already available in a robust, parallelized form, I think that would be sufficient.
