More precision of TotalSegmentator?

Support
1

Hi,
Is there some way to get better resolution when using TotalSegmentator (less than the default resolution of 1.5mm)? Or is there any other program?

1 Like
2

I believe the MONAI Label full body CT model trained on the TotalSegmentator data operates at native CT resolution. @diazandr3s should be able to say for sure.

1 Like
3

Thanks for the ping, @pieper.

@zariliusra this is a good question. The original resolution of the TotalSegmentator dataset is 1.5mm. You could potentially resample volumes and labels to better resolution, but that will modify the labels and might need to be rechecked by an expert again.

Copied from the TotalSegmentator paper (Page 3, first paragraph):

All images were resampled to 1.5 mm isotropic resolution.

MONAI model was trained at 2 resolutions: original (1.5mm) and low resolution (3mm) https://github.com/Project-MONAI/model-zoo/tree/dev/models/wholeBody_ct_segmentation#high-resolution-and-low-resolution-models

Here you see the spacing set for those two models: https://github.com/Project-MONAI/model-zoo/blob/dev/models/wholeBody_ct_segmentation/configs/inference.json#L16

4

So the answer to the good question is - No - without complete new training and investing in better hardware? This is an important topic because the resolution is borderline.

2 Likes
5

Hi @rbumm,

I agree. The TotalSegmentator dataset’s resolution is borderline and a limitation to accurately segmenting tumours and/or small organs. Many CT volumes in real scenarios are 1mm or even bigger (i.e. 0.8mm, 0.4mm).

If we want to train a model using a dataset with a bigger resolution (1mm, 0.8mm, etc.), we’ll need more computing resources to have one model segmenting all segments at once OR train multiple models segmenting fewer segments (10 segments per model)

If we have a verified upsampled dataset, we can retrain the MONAI Label models - no need to start from scratch.

Hope this makes sense.

1 Like
6

I also agree, the main limitation is the dataset’s current resolution. With a higher-res dataset, one could train a higher-res model. Of course, that would benefit from better compute, especially memory: a 32-bit float volume at 512x512x512 and 104 regions requires ~52 GB VRAM just to hold the inference result - alternatively, one could split the labels (as Andres said, and as it was done with nnunet on TotalSegmentator v1).

The TotalSegmentator dataset is an amazing resource, I think it might be even repurposed to create a high-res model, at least to pre-train it. My take on this:

  • High-quality image upsampling: (e.g. from 1.5mm to e.g. 1mm or even more). For interpolation, one could use at least bicubic or Lanczos, perhaps even deep learning superresolution (not sure whether there is a robust model out there).
  • High-quality label upsampling: Instead of using nearest-neighbor interpolation to upsample the labelmaps, I would recommend using SimpleITK with the interpolator sitkLabelGaussian - for me, this always works wonders compared nearest-neighbors.

Something like this function, but replacing the sitkNearestNeighbor interpolator for labels to:

def resample_img(itk_image, out_spacing=[1.0, 1.0, 1.0], is_label=False):
    resample = sitk.ResampleImageFilter()
    resample.SetOutputSpacing(out_spacing)
    ...
    if is_label:
        resample.SetInterpolator(sitk.sitkLabelGaussian)
    ...

It would probably be best to run such pre-processing once to create a HD version of the TotalSegmentator dataset. To train e.g. a model at 1mm^3 resolution, the network input of the wholeBody_ct_segmentation model in the Model Zoo would need to be increased from 96x96x96 to 144x144x144. That is a lot (especially for 105+ output labels), but not impossible with current GPUs.

But training at this scale is easier said than done. Just out of curiosity, what image resolution would be a good “aim”? Are large-FOV CT scans often done at 0.4mm or even 0.8mm?? Would 1mm be a good target resolution?

2 Likes
7

For most use cases, the 1.5mm resolution of the segmentation is good enough. You often have to work with lower-resolution input images anyway. Discussion about improving the resolution only makes sense if you have a specific goal in mind, as creating perfect segmentations (that are suitable for every possible applications) is impossible. What is your clinical application?

Note that if the goal is to just to create nicer visualization then surface rendering is probably not a good approach anyway. Segmentation-enhanced volume rendering has much bigger potential for visualization - see for example in this topic.

1 Like