Novice Guidance for Using MONAI Label to Create an Automatic Segmentation App

Support
, , , ,
1

I attended the recent MONAI Label Workshop, and with a background of sports medicine/orthopaedics research, I am new to this concept and do not have a strong background in computer programming. I am familiar with 3D Slicer, however, and I am looking for additional assistance or resources to help guide my learning.

Essentially, the goal is to create a MONAI Label app that automatically segments the bones of the shoulder joint, primarily, to display a visualization of glenohumeral bone loss to assist with research, analysis, and surgical planning. I have manually segmented several shoulder MRIs, and I am looking to use this work to contribute to a learning model, for efficiency purposes.

Currently, listed are some of the resources that have been helpful in progressing this idea, however, there are some limitations and questions that I was hoping to find clarification for.

Resources:

-Steps for Running a MONAI Label server: GitHub - Project-MONAI/MONAILabel: MONAI Label is an intelligent open source image labeling and learning tool.
-MONAI Label Bootcamp: MONAILabel_MONAI_Bootcamp_Sept_2021 - Google Slides
-MONAI Label-Training from Scratch: MONAI Label - Training from Scratch - YouTube

Limitations/Questions:

-My current computer is a MacBook Pro. As, to my understanding, a Windows or Linux system is required for MONAI Label, what are my options? Would running Windows or Linux on a Mac be sufficient, or would a specific computer be required? Is the purpose of using AWS AppStream to connect to a server capable of this process, if a local network is not available?
-Using Python: Attempting to follow the installation process, the message ‘SyntaxError: invalid syntax’ is listed when entering any of the information into 3D Slicer. Is this because it is being entered on a Mac, or is it simply being entered incorrectly?

Please feel free to let me know if any other information is requested. Thanks!

1 Like
2

Hi -

It sounds like you are on the right track but yes, there are several details to work out. Maybe a good approach would be to sign up for next week’s project week (you can join online, but you’d need to be able to invest some time) and either sign up for a project similar to your plans or create your own page with details (ideally some sample data) and probably people with similar interests can help you sort out some of these details.

2 Likes
3

You can run 3D Slicer and the MONAILabel extension on a Macbook Pro, but should be running the MONAILabel server on a system with CUDA and GPU processing avalable which will make processing, inference (show the results of your trained model in a new dataset) and training itself faster.

A specific computer acting as your MONAILabel Server would be required. See above.

Yes. But any other CUDA-enabled system in your network will also work.

Where exactly do you enter the information you are referring to?

2 Likes
4

Thank you both for the suggestions and information. I am not able to participate in project week, however, I am following some of the posts from Discord to help educate myself and continue learning.

Regarding the question, the information is entered in the Python Console window in 3D Slicer, or the IDLE Shell that was downloaded. Following the installation guide, for example, the message ‘SyntaxError: invalid syntax’ is displayed when ‘pip install monailabel -U’ is entered.

5

The steps for setting up MONAILabel on a Mac are as follows:

  1. Install the MONAILabel extension in Slicer
  2. Connect to a MONAILabel server

Note: The “pip install MONAILabel -U” command should be entered on the system where you want to install the MONAILabel server. The server should have CUDA and a NVIDIA GPU and be reachable over the network. This could be a Google Cloud computer, an AWS EC2 or Appstream instance, or any other server of your choice.

6

Thank you. The MONAILabel extension is installed in Slicer, however, I do not seem to be understanding how to connect to a server. When entering the “pip install MONAILabel -U” command on the desired system, an error message appears.

ERROR: Cannot install monailabel==0.4.0, monailabel==0.4.1, monailabel==0.4.2, monailabel==0.5.0, monailabel==0.5.1, monailabel==0.5.2 and monailabel==0.6.0 because these package versions have conflicting dependencies.

The conflict is caused by:
monailabel 0.6.0 depends on torch>=1.7
monailabel 0.5.2 depends on torch>=1.7
monailabel 0.5.1 depends on torch>=1.7
monailabel 0.5.0 depends on torch>=1.7
monailabel 0.4.2 depends on torch>=1.7
monailabel 0.4.1 depends on torch>=1.7
monailabel 0.4.0 depends on torch>=1.7

To fix this you could try to:

  1. loosen the range of package versions you’ve specified
  2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit Dependency Resolution - pip documentation v24.0.dev0

I do not understand what the error is, or how to apply the suggested solutions to the entered information. Please feel free to let me know if any assistance or suggestions regarding a solution is available, or if it should be directed differently.

The system I am using may not be ideal regarding the previously mentioned requirements, however, my understanding is that the server should be able to be established, and that the processing and functionality would be slower. If the message displayed is a result of incompatibility, please feel free to let me know as well.

I appreciate the assistance!

7

You will need to

python -m pip install --upgrade pip setuptools wheel

and

pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu114

on your system first. Here is a full installation guide for Windows 11.

8

Thank you. The Python version installed was outdated, so after resolving that, by following the installation guide, a server was able to be started.

Now that I understand the process of starting a MONAILabel server, the goal is to create an app that automatically segments the humerus, clavicle, and scapula, utilizing a stored repository of previously segmented MRI scans. It appears that the best way to achieve this goal is to follow the MONAILabel-Training from Scratch video, loading the mentioned MRI dataset in place of the lung dataset used in the video, and training a model based off of those segmentations.

Please feel free to let me know if this plan is missing any steps, if any information is being overlooked, or if any other resources or directions may be valuable.

Thank you!

1 Like
9

Hello!

I wanted to reach out with some questions that I thought of as I am learning this process. I am not able to actually attempt the process yet due to unavailable resources, so I am attempting to map out a plan. I apologize if any of the questions would be easily solved after attempting the process.

  • Based on my goal of using previously segmented images to train a model, is the ‘segmentation’ model the best model to utilize? If not, why would the DeepGrow or DeepEdit be chosen alternatively?

  • When preparing the dataset, will multiple multi-label .mrb files suffice if stored in a folder together, or is there a different format that they should be stored as?

  • As part of the stored segmentations, a background segment has already been established prior to loading onto the MONAI Label module. Can that simply be added as a label when adjusting the labels of the model in the ‘configs’ folder?

  • As the seeds would have already been added when the files are loaded, and the ‘grow from seeds’ function has been initialized and applied, as these files are previously segmented, can that be submitted to the server to train the model?

  • When the label is ‘submitted to the server,’ how secure is that? Would the data have the potential to be accessible by an outside source in any way?

Thank you for any answers or assistance in helping me learn this information!

1 Like
10

Very good questions, @JacobD. We will try to answer them step by step, hopefully with the help of @diazandr3s and @pieper.

Last question first:

If you run the server and 3D Slicer with the MONAILabel extension on the same computer: No, that data will not be accessible by an outside source as long as you do not open specific ports in the firewall.

1 Like
11

I am not associated with the MonaiLabel development team but glad to help, if possible. Did my lung segmentation training exercise with the “segmentation” model after starting with DeepEdit and the results were not convincing.

You would need to edit the segmentation.py config file on your system here and add the labels that exactly match the ones you are gonna use.

My workflow was: Open the dataset in 3D Slicer, make a segmentation with MONAILabel segmentation tools or Slicer tools, then “submit to the server” and save it locally as well for backup.
Did this for 5-7 segmentation, train, and after that already got a relatively good inference on the next loaded dataset. Around 20 labeled datasets and several hours (better days) of training are the low limit for getting good inference.

Most AI tools work with NIFTI (*.nii.gz) format and this is the format in which the MONAILabel server will store your labels in its dataset directory.

Do not understand this, please post an image

Yes.

Hope that helps …

12

Thank you, that information is very helpful.

I attempted to edit the segmentation.py config file, and was able to successfully change the label names, so it is helpful to know that that is the correct process.

1058×216 17.1 KB

Regarding the question, my phrasing may not have been clear. The picture is from the Segment Editor module of a segmented file. The segmented files in the dataset were originally segmented into ‘scapula,’ ‘clavicle,’ ‘humerus,’ and ‘other,’ which is essentially the same as ‘background.’ The ‘MONAI Label-Training from Scratch’ Youtube video, however, does not list ‘background’ as a label when editing the config file, it adds it when using the Segment Editor tab of the MONAI Label extension to paint seeds. I did not know if there was a specific reason for following that order.

My workflow would be slightly different since I would not be making any edits to the segmentations after loading them, they would just be opened in slicer and immediately submitted to the server since they are already segmented. When I open files in Slicer, however, I usually drag and drop the locally saved .mrb file. Would I need to go back and convert all of the .mrb files into NIFTI files?

Thanks!

13

No, you can work with your MRB files.

14

BTW have you tried TotalSegmenator for analyzing your data? It should be able to segment the bones you mentioned out of the box.

15

Many thanks for the ping, @rbumm.

Just came back from leave, and I’m catching up on emails/discussions.

@JacobD all these are great questions. I see most of them have been addressed by @rbumm.

Here are my comments on them:

  • Based on my goal of using previously segmented images to train a model, is the ‘segmentation’ model the best model to utilize? If not, why would the DeepGrow or DeepEdit be chosen alternatively?

Yes, I’d recommend starting with the Segmentation model.
DeepGrow and DeepEdit are interactive models. They feature an option of modifying/creating the segments by using clicks. One of the disadvantages of DeepEdit though is that it uses the whole volume to train the model. This means a bigger GPU is needed to have high-quality segmentations.

When preparing the dataset, will multiple multi-label .mrb files suffice if stored in a folder together, or is there a different format that they should be stored as?

For multi-label segmentation, you should merge all segmentations into a single file. Here you can find an example using the Total Segmentator dataset showing how you should prepare the label files for multi-label segmentation: https://youtu.be/KtPE8m0LvcQ?t=500

As part of the stored segmentations, a background segment has already been established prior to loading onto the MONAI Label module. Can that simply be added as a label when adjusting the labels of the model in the ‘configs’ folder?

If using the Segmentation model, you don’t need to specify the background label

As the seeds would have already been added when the files are loaded, and the ‘grow from seeds’ function has been initialized and applied, as these files are previously segmented, can that be submitted to the server to train the model?

Yes, you should be able to submit the segmentation to the MONAI Label model regardless of how they have been created.

When the label is ‘submitted to the server,’ how secure is that? Would the data have the potential to be accessible by an outside source in any way?

As @rbumm nicely pointed out, if you run both server and Slicer on the same PC, no data will go outside the computer. Another situation is when MONAI Label server runs on a different PC - in that case, security will depend on the type of connection you have to that PC.

I hope this helps.

Thanks again, @rbumm

16

Thank you both for the information. Starting with the Segmentation model appears to be a good approach.

Thanks for the direction to the video. To help me understand the dataset preparation slide, would the green files be the original, not segmented scans, and the red files be the single file multi-label segmentations?
image
Separately, would my segmentations already be in a single file if they were created at the same time and saved as an .mrb file together? Would it be best to simply ‘export to label’ the multi-segmented files and store it as .nii.gz if I am not sure how it is currently arranged?

Does this mean that I would have to remove the background label if it was previously added, or would it still function properly if it was already listed?

I do not have much experience using Total Segmentator, but looked into it. I did not think that it would work best for segmenting the specific type of MRI sequence being used, since it associated with CT scans, so it seemed best to train a new model.

Thank you!

1 Like
17

Hi @JacobD,

would the green files be the original, not segmented scans, and the red files be the single file multi-label segmentations?

Green files are the volumes. Red files located in labels/final folder are the ground truth/expert annotations of green files (volumes)

Separately, would my segmentations already be in a single file if they were created at the same time and saved as an .mrb file together? Would it be best to simply ‘export to label’ the multi-segmented files and store it as .nii.gz if I am not sure how it is currently arranged?

Not sure if I understood this correctly. In order to use MONAI Label + Slicer, annotations should be in a single file per volume. Regardless if they are single or multiple annotations per volume.

Does this mean that I would have to remove the background label if it was previously added, or would it still function properly if it was already listed?

You shouldn’t have an issue with the background label

Hope this helps,

18

“I encountered a NameError: name ‘AddChanneld’ is not defined in my MONAI code. I suspect there might be a typo or a misnaming issue. Has anyone experienced a similar error, and can you provide guidance on resolving this issue? The specific line of code causing the error seems to involve the term ‘AddChanneld.’ Should it be ‘AddChannel,’ or is there a specific function or import that I might be missing?”

19

Did you install MonaiLabel as we have recommended here ?
Please provide some system and dataset specifications.
Is the MonaiLabel server starting without errors ? If not please post the complete error message.

20

NameError Traceback (most recent call last)
in <cell line: 16>()
30 ).to(device)
31
—> 32 data_in = prepare(data_dir, cache=True)
33
34 #loss_function = DiceCELoss(to_onehot_y=True, sigmoid=True, squared_pred=True, ce_weight=calculate_weights(1792651250,2510860).to(device))

in prepare(in_dir, pixdim, a_min, a_max, spatial_size, cache)
79 LoadImaged(keys=[“vol”, “seg”]),
80 # EnsureChannelFirst(),
—> 81 AddChanneld(keys=[“vol”, “seg”]),
82 Spacingd(keys=[“vol”, “seg”], pixdim=pixdim, mode=(“bilinear”, “nearest”)),
83 Orientationd(keys=[“vol”, “seg”], axcodes=“RAS”),

NameError: name ‘AddChanneld’ is not defined