# New extension: MONAI Auto3DSeg - raise the bar in AI medical image segmentation

**URL:** <https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680>\
**Category:** Announcements\
**Tags:** feature, segmentation, segmentations, segment-editor, ai, monai, monailabel\
**Created:** [April 23, 2024, 2:21pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680 "2024-04-23T14:21:28Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [April 23, 2024, 2:21pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/1 "2024-04-23T14:21:28Z")

</div>

# New extension: MONAI Auto3DSeg - raise the bar in AI medical image segmentation

MONAI Auto3DSeg extension is a collaboration between MONAI and 3D Slicer developer teams (led by Andres Diaz Pinto - NVIDIA and Andras Lasso - PerkLab) to improve on the state-of-the-art in fully automatic 3D medical image segmentation and make the results widely accessible.

The extension comes with dozens of pre-trained segmentation models for specific clinical use cases, which are designed to be fast and run anywhere within minutes - on any average computer, without GPU, even on laptops. The models can segment images of various modalities (CT, MRI), anatomies (mediastinal, vertebra, brain, prostate, lungs, etc.) and pathologies (tumor, hemorrhage, edema, etc.), using one or more input images. All models come with a description and sample data set for easy testing. See [complete list of models - with screenshots, computation times, list of segments](https://github.com/lassoan/SlicerMONAIAuto3DSeg/releases/tag/ModelsTestResults).

**To get started** : [install the latest Slicer Preview Release (revision 32818 or later) and the MONAIAuto3DSeg extension](https://github.com/lassoan/SlicerMONAIAuto3DSeg/tree/main?tab=readme-ov-file#setup) and follow this [step-by-step tutorial](https://github.com/lassoan/SlicerMONAIAuto3DSeg?tab=readme-ov-file#setup). 3-minute video demo/tutorial showing the module in action:

[![](https://img.youtube.com/vi/N8Dy2Bu8fjo/maxresdefault.jpg "Automatic multi modality AI medical image segmentation in 3D Slicer using MONAIAuto3DSeg extension") ](https://www.youtube.com/watch?v=N8Dy2Bu8fjo)

MONAI Auto3DSeg allows [adding your own custom models](https://github.com/lassoan/SlicerMONAIAuto3DSeg/tree/main?tab=readme-ov-file#developers) to the plugin. Users can leverage this feature to create segmentation models that are optimized for their own data, to their specific clinical requirements.

The extension works offline, without internet connection (after the setup is completed and selected model is downloaded) and does not send any data to the cloud or any other computer.

The MONAI Auto3DSeg software is open-source and freely accessible (Apache License 2.0). The developers do not claim that the tools are appropriate for any specific clinical purpose and the user is responsible for obtaining any necessary ethics or regulatory approvals.

---

<div class="post-metadata">

**Author:** ![dzenanz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/dzenanz/32/1992_2.png) [@dzenanz](https://discourse.slicer.org/u/dzenanz)\
**Post date:** [April 23, 2024, 2:46pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/2 "2024-04-23T14:46:41Z")

</div>

Does it work on Windows? Auto3DSeg [didn’t work on Windows](https://github.com/Project-MONAI/MONAI/issues/6821#issuecomment-1789933361) half a year ago.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 23, 2024, 2:48pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/3 "2024-04-23T14:48:26Z")

</div>

Yes, it works well on Windows (also on Linux and macOS). Everything is automatically installed and set up by a single click when the user starts the segmentation.

---

<div class="post-metadata">

**Author:** ![dzenanz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/dzenanz/32/1992_2.png) [@dzenanz](https://discourse.slicer.org/u/dzenanz)\
**Post date:** [April 23, 2024, 3:58pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/4 "2024-04-23T15:58:55Z")

</div>

I now see that this extension only runs the inference, and not training.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 23, 2024, 4:09pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/5 "2024-04-23T16:09:22Z")

</div>

Yes, the extension provides lots of ready-to use pre-trained models for users. They cover a wide range of body parts, anatomical structures and lesions, using multiple imaging modalities.

If a developer wants to create new models, there are links to examples and tutorials in the [documentation](https://github.com/lassoan/SlicerMONAIAuto3DSeg?tab=readme-ov-file#training-models). For interactive training, [integration with MONAILabel](https://github.com/Project-MONAI/MONAILabel/discussions/1630) is being considered. But so far models have been trained from already labeled data, so it was all done by just running a Python script non-interactively.

---

<div class="post-metadata">

**Author:** ![ShaneKirkbride](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/shanekirkbride/32/11518_2.png) [@ShaneKirkbride](https://discourse.slicer.org/u/ShaneKirkbride)\
**Post date:** [April 23, 2024, 4:49pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/6 "2024-04-23T16:49:37Z")

</div>

Has any work been done related to cardiology/heart segmentation?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 23, 2024, 4:54pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/7 "2024-04-23T16:54:10Z")

</div>

Yes, there are a couple of heart segmentation models. See complete list of models [here](https://github.com/lassoan/SlicerMONAIAuto3DSeg/releases/tag/ModelsTestResults). New models will be added based on user demand and availability of training data.

The SlicerHeart group is developing models for cardiac ultrasound segmentation using MONAI Auto3DSeg and it seems to work well, too, but I’m not sure if those models will be made openly available. If they will be then we’ll add them to this extension.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [April 23, 2024, 8:06pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/8 "2024-04-23T20:06:42Z")

</div>

Looks very cool, thank you for putting this together! 👍

What are the terms of the licenses for the models that are integrated?

From what I have seen before, sometimes the license imposes restrictions on the use of the segmentation results produced by the model. It would be helpful if the [aforementioned list of models](https://github.com/lassoan/SlicerMONAIAuto3DSeg/releases/tag/ModelsTestResults) included licensing information.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [April 23, 2024, 8:20pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/9 "2024-04-23T20:20:18Z")

</div>

To clarify, I do understand that the code places no restrictions on reuse, but I believe in some cases model may be covered by a different license, or model license terms may be dictated by the license covering the data that was used to train the model.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 23, 2024, 9:01pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/10 "2024-04-23T21:01:27Z")

</div>

@diazandr3s can provide more details on the training data sources, but in short: we only trained on data sets that did not impose any restrictions on how the data is used. Previously, we used some data sets that were distributed with restrictive licenses, but in the end we decided to drop all those models, both to simplify our life (so that we don’t need to track what models are restricted and how and warn the users, etc.) and also not promote any data sets that come with strings attached.

We will add more information on all sources that were used to train each model to make this more clear.

---

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [April 23, 2024, 9:12pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/11 "2024-04-23T21:12:44Z")

</div>

Thanks for asking this, @fedorov.

This is a very good point. We’ll add that information to all the available models.

To answer your question now, we’ve used BRATS (You are free to use and/or refer to the BraTS datasets in your own research …), Medical Segmentation Decathlon (Creative Commons license CC-BY-SA4.0), TotalSegmentatorV1 (Apache License 2.0) and TotalSegmentatorV2 (Apache License 2.0) datasets to train the available models.

The model for ICrH and brain CT segmentation was trained on a randomized controlled trial conducted by Ken: [Add codes for intracranial hemorrhage model by lassoan · Pull Request #40 · lassoan/SlicerMONAIAuto3DSeg · GitHub](https://github.com/lassoan/SlicerMONAIAuto3DSeg/pull/40#issuecomment-2050509368)

We’ll update all this in the repo.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [April 23, 2024, 10:05pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/12 "2024-04-23T22:05:56Z")

</div>

@lassoan and @diazandr3s thanks for the quick responses! Having this information along the models will be great.

> [@diazandr3s](#):
>
> Medical Segmentation Decathlon (Creative Commons license CC-BY-SA4.0)

I think CC-BY-SA may be equivalent to the viral GNU - “SA” means “Share Alike”: [CC BY-SA 4.0 Deed | Attribution-ShareAlike 4.0 International | Creative Commons](https://creativecommons.org/licenses/by-sa/4.0/deed.en).

> [@diazandr3s](#):
>
> TotalSegmentatorV1 (Apache License 2.0) and TotalSegmentatorV2 (Apache License 2.0)

From what I can tell, TotalSegmentator datasets are shared under [CC-BY](https://creativecommons.org/licenses/by-sa/4.0/deed.en), which is less restrictive than CC-BY-SA - it only requires attribution: [Dataset with segmentations of 117 important anatomical structures in 1228 CT images](https://zenodo.org/records/10047292).

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [April 23, 2024, 10:22pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/13 "2024-04-23T22:22:41Z")

</div>

> [@fedorov](#):
>
> equivalent to the viral GNU

Yes, [according to the CC site](https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses/#:~:text=GPLv3%3A%20The%20GNU%20General%20Public%20License%20version%203%20was%20declared%20a%20%E2%80%9CBY%2DSA%E2%80%93Compatible%20License%E2%80%9D%20for%20version%204.0%20on%208%20October%202015.) GPLv3 is one of the only licenses you can use for material based on CC-BY-SA content. Just how this applies to the trained models is [a tricky topic](https://hbr.org/2023/04/generative-ai-has-an-intellectual-property-problem).

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 23, 2024, 10:34pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/14 "2024-04-23T22:34:09Z")

</div>

The main thing we wanted to make sure is that we are not tainted by non-commercial clauses (NC). Even if share-alike (SA) is interpreted the strictest possible way, it is still allowed to commercially use models that were trained on SA data if the resulting model is shared, which it is.

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [April 23, 2024, 11:13pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/15 "2024-04-23T23:13:37Z")

</div>

> [@lassoan](#):
>
> it is still allowed to commercially use models that were trained on SA data if the resulting model is shared, which it is.

Right - I agree the models offered here are compatible with the SA terms, but if someone trains a new model based on this data or on data generated by models trained on this data and wishes not to share it then they should review these terms with IP consultation.

---

<div class="post-metadata">

**Author:** ![amitjc](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/amitjc/32/16760_2.png) [@amitjc](https://discourse.slicer.org/u/amitjc)\
**Post date:** [April 24, 2024, 2:49am UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/16 "2024-04-24T02:49:52Z")

</div>

This is awesome. Thanks a ton.

Amitkumar J. Choudhari, DNB, FRCR  
Consultant Radiologist & Associate Prof.,  
Tata Memorial Centre.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 25, 2024, 2:51pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/17 "2024-04-25T14:51:13Z")

</div>

12 posts were split to a new topic: [Facial muscle segmentation using MONAI Auto3DSeg](https://discourse.slicer.org/t/facial-muscle-segmentation-using-monai-auto3dseg/35734)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 25, 2024, 6:47pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/28 "2024-04-25T18:47:04Z")

</div>

5 posts were split to a new topic: [Error when running MONAI Auto3DSeg (cannot import name ‘MetaKeys’ from ‘monai.utils’)](https://discourse.slicer.org/t/error-when-running-monai-auto3dseg-cannot-import-name-metakeys-from-monai-utils/35738)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 29, 2024, 1:46pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/29 "2024-04-29T13:46:49Z")

</div>

3 posts were split to a new topic: [Help with setting up MONAILabel](https://discourse.slicer.org/t/help-with-setting-up-monailabel/35796)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [May 11, 2024, 12:27pm UTC](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680/30 "2024-05-11T12:27:21Z")

</div>

5 posts were split to a new topic: [Automatic segmentation tools for brain AVM?](https://discourse.slicer.org/t/automatic-segmentation-tools-for-brain-avm/36082)

[Next page](https://discourse.slicer.org/t/new-extension-monai-auto3dseg-raise-the-bar-in-ai-medical-image-segmentation/35680.md?page=2)