New extension: MONAI Auto3DSeg - raise the bar in AI medical image segmentation
MONAI Auto3DSeg extension is a collaboration between MONAI and 3D Slicer developer teams (led by Andres Diaz Pinto - NVIDIA and Andras Lasso - PerkLab) to improve on the state-of-the-art in fully automatic 3D medical image segmentation and make the results widely accessible.
The extension comes with dozens of pre-trained segmentation models for specific clinical use cases, which are designed to be fast and run anywhere within minutes - on any average computer, without GPU, even on laptops. The models can segment images of various modalities (CT, MRI), anatomies (mediastinal, vertebra, brain, prostate, lungs, etc.) and pathologies (tumor, hemorrhage, edema, etc.), using one or more input images. All models come with a description and sample data set for easy testing. See complete list of models - with screenshots, computation times, list of segments.
MONAI Auto3DSeg allows adding your own custom models to the plugin. Users can leverage this feature to create segmentation models that are optimized for their own data, to their specific clinical requirements.
The extension works offline, without internet connection (after the setup is completed and selected model is downloaded) and does not send any data to the cloud or any other computer.
The MONAI Auto3DSeg software is open-source and freely accessible (Apache License 2.0). The developers do not claim that the tools are appropriate for any specific clinical purpose and the user is responsible for obtaining any necessary ethics or regulatory approvals.
Yes, it works well on Windows (also on Linux and macOS). Everything is automatically installed and set up by a single click when the user starts the segmentation.
Yes, the extension provides lots of ready-to use pre-trained models for users. They cover a wide range of body parts, anatomical structures and lesions, using multiple imaging modalities.
If a developer wants to create new models, there are links to examples and tutorials in the documentation. For interactive training, integration with MONAILabel is being considered. But so far models have been trained from already labeled data, so it was all done by just running a Python script non-interactively.
Yes, there are a couple of heart segmentation models. See complete list of models here. New models will be added based on user demand and availability of training data.
The SlicerHeart group is developing models for cardiac ultrasound segmentation using MONAI Auto3DSeg and it seems to work well, too, but Iām not sure if those models will be made openly available. If they will be then weāll add them to this extension.
Looks very cool, thank you for putting this together!
What are the terms of the licenses for the models that are integrated?
From what I have seen before, sometimes the license imposes restrictions on the use of the segmentation results produced by the model. It would be helpful if the aforementioned list of models included licensing information.
To clarify, I do understand that the code places no restrictions on reuse, but I believe in some cases model may be covered by a different license, or model license terms may be dictated by the license covering the data that was used to train the model.
@diazandr3s can provide more details on the training data sources, but in short: we only trained on data sets that did not impose any restrictions on how the data is used. Previously, we used some data sets that were distributed with restrictive licenses, but in the end we decided to drop all those models, both to simplify our life (so that we donāt need to track what models are restricted and how and warn the users, etc.) and also not promote any data sets that come with strings attached.
We will add more information on all sources that were used to train each model to make this more clear.
This is a very good point. Weāll add that information to all the available models.
To answer your question now, weāve used BRATS (You are free to use and/or refer to the BraTS datasets in your own research ā¦), Medical Segmentation Decathlon (Creative Commons license CC-BY-SA4.0), TotalSegmentatorV1 (Apache License 2.0) and TotalSegmentatorV2 (Apache License 2.0) datasets to train the available models.
Yes, according to the CC site GPLv3 is one of the only licenses you can use for material based on CC-BY-SA content. Just how this applies to the trained models is a tricky topic.
The main thing we wanted to make sure is that we are not tainted by non-commercial clauses (NC). Even if share-alike (SA) is interpreted the strictest possible way, it is still allowed to commercially use models that were trained on SA data if the resulting model is shared, which it is.
Right - I agree the models offered here are compatible with the SA terms, but if someone trains a new model based on this data or on data generated by models trained on this data and wishes not to share it then they should review these terms with IP consultation.
