What is the difference between using MONAI Auto3DSeg and TotalSegmentor for automatic segmentation? There is also TotalSegmentator in MONAI Auto3DSeg.
1 Like
That’s a good point and I can see why there’s confusion. TotalSegmentator refers to both the ground-truth dataset and the nnU-Net based model trained on that data. Since the data was released freely, the team working on MONAI Auto3DSeg also trained a model using that data. For most purposes they should give very similar results, although the memory usage and GPU efficiency might be different.
I don’t think anyone has done an exhaustive comparison, and in fact both tools are evolving rapidly so you should maybe try both and see what works best for you. Report back if you learn anything interesting.
1 Like
Hi @khajta,
that’s a great question. I’m gonna answer it a bit more detailed manner, perhaps this will also help others who have questions in this direction. Pinging also @diazandr3s for his opinion. And I just saw that @pieper just posted an answer as well, much more concise, but I already typed everything, so I’m also gonna send it 
First of all, TotalSegmentator is two things (as Steven said): a dataset and a model. Important to keep the 2 apart.
When TotalSegmentator came out, the authors released both the data and the pre-trained model under the same name. The model was trained with nnUnet - in case you’re not familiar with it, it’s an automatic model training framework, highly sophisticated and still state-of-the-art. nnUnet is a custom framework based on PyTorch which was built by researchers from German Cancer Research Center (DKFZ)/Helmholtz Research.
The dataset was also used to train a model with Auto3DSeg. This tool is also an automatic model training framework, but built on top of the MONAI ecosystem. It was inspired by nnUnet and fulfills a similar purpose, but it is using native MONAI transforms, dataloaders, models, training/inference workflows etc.To my knowledge, there is no open-source model yet which was trained with Auto3DSeg on TotalSegmentator data. If you know one, please send me the link, happy to look into it.
However, there is a model which was trained with MONAI Label on TotalSegmentator data. This model is available in the MONAI Model Zoo, it is called wholeBody_ct_segmentation. MONAI Label is a third tool which you can freely use. It’s purpose is slightly different: while Auto3DSeg and nnUnet assume that you already have annotated data (images+segmentations), MONAI Label assumes that you only very few segmentations. It provides you with an online model training framework, with which you can simultaneously annotate data in batches, and incrementally train a model to get better and better at supporting you during the annotation. Important to note: the purpose of MONAI Label is not to train models that achieve challenge-winning performance - this would require cross-validation, ensembling, HPO etc (these things are done under the hood in nnUnet/Auto3DSeg). The focus is rather interactive annotation via integration into UI frontends like Slicer. But the models are still good enough to help you annotate.
If you’re asking which model/tool to use:
- nnUnet: Allows you to train and run inference on state-of-the-art models. You can get to the level of challenge-winning accuracy with very little effort. It assumes you have annotated training data. The official model trained on and shipped with the TotalSegmentator data is the reference model, and should be used for comparisons and benchmarks.
- Auto3DSeg: Another auto-training framework, with similar performance and ease of use as nnUnet, but natively in the MONAI ecosystem (and as Steven said, different optimization of memory/GPU usage). To my knowledge, there’s no open-source model yet trained on TotalSegmentator data, but given sufficient compute, it would be easy to do so.
- MONAI Label: Great tool to help you use AI assistance to interactively annotate your data, or quickly train models and view their performance in 3D Slicer. The model performance will likely not reach the quality of nnUnet/Auto3DSeg model ensembles though.
By the way, you don’t even have to decide between nnUnet and MONAI - there’s a MONAI wrapper of the nnUnet API, which allows you to trigger training and inference of nnUnet models directly from MONAI.
4 Likes
Very nice summary
Apparently, a Auto3DSeg model was indeed trained with TotalSegmentator data
1 Like
I have one more question. If I have CT dataset and I would like the to obtain the organ segmentation information for evaluating the absorbed dose, is it the same result using nnUnet, Auto3DSeg, or MONAI Label?
1 Like
What I meant in my answer above is that I don’t think anyone has researched that yet. If the answer is important to you, then you should run the experiment. Please report back what you learn.
1 Like
Good question, @khajta.
And great answers already from @pieper and @mangotee.
I’d like to comment on this part:
To my knowledge, there is no open-source model yet which was trained with Auto3DSeg on TotalSegmentator data. If you know one, please send me the link, happy to look into it.
We trained the Autro3DSeg on both TotalSegmentator V1 and TotalSegmentator V2 and made available on the Slicer MONAI Auto3DSeg: GitHub - lassoan/SlicerMONAIAuto3DSeg: Extension for 3D Slicer for running MONAI Auto3DSeg models
Please download the latest Slicer and install the Auto3DSeg via the Extension Manager.
It’d be great to hear your thoughts on these models 
Hope this helps,
Hi! Taking advantage of what you have mentioned about this, both @pieper and @mangotee , I wanted to ask you a question.
First, I want to say that I don’t know much about this topic. But this question came to mind and I wanted to ask you.
Why are the segmentation results obtained from both TotalSegmentator and Monai Auto3dSeg not completely identical? I understand that if they are trained with the same data, they should offer identical results… But as I said, this topic is not my strong suit and maybe I am wrong.
I have been testing both, and there is a very small variation, but there is a variation in the results.
I look forward to your response, and thank you in advance.
1 Like
These are two different models (nnunet/segresnet), trained on not exactly the same data, so it is pretty amazing that they provide so similar results.
1 Like