New extension: MONAI Auto3DSeg - raise the bar in AI medical image segmentation

Announcements
, , , , , ,
1

New extension: MONAI Auto3DSeg - raise the bar in AI medical image segmentation

MONAI Auto3DSeg extension is a collaboration between MONAI and 3D Slicer developer teams (led by Andres Diaz Pinto - NVIDIA and Andras Lasso - PerkLab) to improve on the state-of-the-art in fully automatic 3D medical image segmentation and make the results widely accessible.

The extension comes with dozens of pre-trained segmentation models for specific clinical use cases, which are designed to be fast and run anywhere within minutes - on any average computer, without GPU, even on laptops. The models can segment images of various modalities (CT, MRI), anatomies (mediastinal, vertebra, brain, prostate, lungs, etc.) and pathologies (tumor, hemorrhage, edema, etc.), using one or more input images. All models come with a description and sample data set for easy testing. See complete list of models - with screenshots, computation times, list of segments.

To get started: install the latest Slicer Preview Release (revision 32818 or later) and the MONAIAuto3DSeg extension and follow this step-by-step tutorial. 3-minute video demo/tutorial showing the module in action:

MONAI Auto3DSeg allows adding your own custom models to the plugin. Users can leverage this feature to create segmentation models that are optimized for their own data, to their specific clinical requirements.

The extension works offline, without internet connection (after the setup is completed and selected model is downloaded) and does not send any data to the cloud or any other computer.

The MONAI Auto3DSeg software is open-source and freely accessible (Apache License 2.0). The developers do not claim that the tools are appropriate for any specific clinical purpose and the user is responsible for obtaining any necessary ethics or regulatory approvals.

21 Likes
2

Does it work on Windows? Auto3DSeg didnā€™t work on Windows half a year ago.

3

Yes, it works well on Windows (also on Linux and macOS). Everything is automatically installed and set up by a single click when the user starts the segmentation.

3 Likes
4

I now see that this extension only runs the inference, and not training.

5

Yes, the extension provides lots of ready-to use pre-trained models for users. They cover a wide range of body parts, anatomical structures and lesions, using multiple imaging modalities.

If a developer wants to create new models, there are links to examples and tutorials in the documentation. For interactive training, integration with MONAILabel is being considered. But so far models have been trained from already labeled data, so it was all done by just running a Python script non-interactively.

1 Like
6

Has any work been done related to cardiology/heart segmentation?

1 Like
7

Yes, there are a couple of heart segmentation models. See complete list of models here. New models will be added based on user demand and availability of training data.

The SlicerHeart group is developing models for cardiac ultrasound segmentation using MONAI Auto3DSeg and it seems to work well, too, but Iā€™m not sure if those models will be made openly available. If they will be then weā€™ll add them to this extension.

1 Like
8

Looks very cool, thank you for putting this together! :+1:

What are the terms of the licenses for the models that are integrated?

From what I have seen before, sometimes the license imposes restrictions on the use of the segmentation results produced by the model. It would be helpful if the aforementioned list of models included licensing information.

1 Like
9

To clarify, I do understand that the code places no restrictions on reuse, but I believe in some cases model may be covered by a different license, or model license terms may be dictated by the license covering the data that was used to train the model.

1 Like
10

@diazandr3s can provide more details on the training data sources, but in short: we only trained on data sets that did not impose any restrictions on how the data is used. Previously, we used some data sets that were distributed with restrictive licenses, but in the end we decided to drop all those models, both to simplify our life (so that we donā€™t need to track what models are restricted and how and warn the users, etc.) and also not promote any data sets that come with strings attached.

We will add more information on all sources that were used to train each model to make this more clear.

3 Likes
11

Thanks for asking this, @fedorov.

This is a very good point. Weā€™ll add that information to all the available models.

To answer your question now, weā€™ve used BRATS (You are free to use and/or refer to the BraTS datasets in your own research ā€¦), Medical Segmentation Decathlon (Creative Commons license CC-BY-SA4.0), TotalSegmentatorV1 (Apache License 2.0) and TotalSegmentatorV2 (Apache License 2.0) datasets to train the available models.

The model for ICrH and brain CT segmentation was trained on a randomized controlled trial conducted by Ken: Add codes for intracranial hemorrhage model by lassoan Ā· Pull Request #40 Ā· lassoan/SlicerMONAIAuto3DSeg Ā· GitHub

Weā€™ll update all this in the repo.

1 Like
12

@lassoan and @diazandr3s thanks for the quick responses! Having this information along the models will be great.

I think CC-BY-SA may be equivalent to the viral GNU - ā€œSAā€ means ā€œShare Alikeā€: CC BY-SA 4.0 Deed | Attribution-ShareAlike 4.0 International | Creative Commons.

From what I can tell, TotalSegmentator datasets are shared under CC-BY, which is less restrictive than CC-BY-SA - it only requires attribution: Dataset with segmentations of 117 important anatomical structures in 1228 CT images.

1 Like
13

Yes, according to the CC site GPLv3 is one of the only licenses you can use for material based on CC-BY-SA content. Just how this applies to the trained models is a tricky topic.

1 Like
14

The main thing we wanted to make sure is that we are not tainted by non-commercial clauses (NC). Even if share-alike (SA) is interpreted the strictest possible way, it is still allowed to commercially use models that were trained on SA data if the resulting model is shared, which it is.

2 Likes
15

Right - I agree the models offered here are compatible with the SA terms, but if someone trains a new model based on this data or on data generated by models trained on this data and wishes not to share it then they should review these terms with IP consultation.

1 Like
16

This is awesome. Thanks a ton.

Amitkumar J. Choudhari, DNB, FRCR
Consultant Radiologist & Associate Prof.,
Tata Memorial Centre.

1 Like
17

12 posts were split to a new topic: Facial muscle segmentation using MONAI Auto3DSeg

28

5 posts were split to a new topic: Error when running MONAI Auto3DSeg (cannot import name ā€˜MetaKeysā€™ from ā€˜monai.utilsā€™)

29

3 posts were split to a new topic: Help with setting up MONAILabel

30

5 posts were split to a new topic: Automatic segmentation tools for brain AVM?