# How to start with monailabel for new models

**URL:** <https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063>\
**Category:** Support\
**Tags:** monailabel\
**Created:** [December 14, 2021, 10:55pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063 "2021-12-14T22:55:54Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 25, 2022, 8:03am UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/42 "2022-05-25T08:03:22Z")

</div>

> [@diazandr3s](#):
>
> I meant the radiologist/clinician using MONAI Label

@diazandr3s or that use case: How would the annotator benefit from monailabel and the monailabel model during segmenting without labels? Which buttons could he press or which functions would he use to create AI-assisted segmentation easier/faster than in the normal segment editor?

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [May 25, 2022, 2:32pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/43 "2022-05-25T14:32:31Z")

</div>

> [@rbumm](#):
>
> How would the annotator benefit from monailabel and the monailabel model during segmenting without labels?

MonaiLabel can be used both to infer and train. If your goal is to facilitate segmentation, first you train a model in say a small number of samples with labelmaps. You might even get something with 10-12 samples (not good but ok).

For inference you have two options, you can either use a dataset loaded into Slicer scene and can ask MonaiLabel to segment it. Or you can deploy the MonaiLabel server such that there are unsegmented volumes on the server side and that’s displayed in the dropdown menu in the monailabel extension and you can just load one after other and run inference on them.

Then there is even an option to push those segmentations back to the server and restart training with them (in addition to the first 10-12 samples you have trained the model on).

The UI is a bit complex, and as I mentioned to the @diazandr3s the inference and training option probably should be decoupled (mostly to avoid accidentally running the training). This might be done perhaps at the server side (e.g., a flag to deploy the server only for inference, or some sort of authentication for training)

---

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [May 25, 2022, 10:34pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/44 "2022-05-25T22:34:14Z")

</div>

Thanks for the detailed explanation, @muratmaga

@rbumm please let us know if there is anything else I can help with 🙂

All your feedback is valuable to making MONAI Label better and easier to use.

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 26, 2022, 1:10pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/45 "2022-05-26T13:10:24Z")

</div>

> [@muratmaga](#):
>
> MonaiLabel can be used both to infer and train. If your goal is to facilitate segmentation, first you train a model in say a small number of samples with labelmaps. You might even get something with 10-12 samples (not good but ok).

Thank you. Hope it is ok for the community to push that thread up again, if not, I could also DM @diazandr3s or @muratmaga.

Probably best to try the train option. Is it correct that I would need to

- segment each of let’s say 12 of my 65 lung datasets and create “right lung”, “left lung” and “airways”
- am I allowed to do this segmentation in a different segmentation program?
- after segmentation of each dataset → switch back to monailabel extension and “submit Label”
- After each “Submit” or after a few “Submits” do a “Train”

Now comes the part which I do not understand:

At patient 12/65 I would like to check the performance of the trained monailabel model.  
So I load patient dataset 13.  
Expect to press a button in monailabel which makes the extension populate the labels “right lung”, left lung", “airways” in dataset 13 from what it has learned so far in patients 1-12. I could then improve and “Submit” again.

- Is there such a button / menu entry / python command in monailabel?
- Do I have to place any markups/seeds for this?

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [May 26, 2022, 1:18pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/46 "2022-05-26T13:18:36Z")

</div>

> [@rbumm](#):
>
> Thank you. Hope it is ok for the community to push that thread up again, if not, I could also DM @diazandr3s or @muratmaga.

Let’s do keep this public - I find it very valuable and I hope everyone else does too! It’s really important to learn, get feedback, and as needed improve the process or interface.

Your description of the workflow sounds right to me, and your expectation that 12 or 65 datasets should give a first pass model sounds consistent with expectations I have heard expressed.

> [@rbumm](#):
>
> - Is there such a button / menu entry / python command in monailabel?
> - Do I have to place any markups/seeds for this?

For me, once I had the model trained to 75-80% I was able to use the Run. button to apply to a new case (either one of the training set or a new similar volume loaded in Slicer). No new seeds required.

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/7/5/75b564a075462f3a5d7db43b7aa863c58d4df224.png)

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [May 26, 2022, 3:11pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/47 "2022-05-26T15:11:01Z")

</div>

I will list my experience and hopefully @diazandr3s can chime in with more technical aspects.

1. You can do the segmentations in any program, provided that you can export a labelmap and that labelmap and associated volume line up correctly in Slicer.
2. Sounds like you want to do multi-label segmentation. If that’s the case you really want to pay a lot of attention that each structure is label with identical label in each of the labels. If Left Lung is 2, and Right Lung is 3 in some segments and vice versa your training will fail.
3. I haven’t used the Active Learning paradigm yet, but what you describe sounds reasonable (meaning you push newly segmented volumes and associated labelmaps back to the MonaiServer, and then when you have a bunch more, retrain the model).
4. For how to parse the data automatically, you need to look at the data organization on the MonaiLabel server side. As I recall, you will organize such that 65 volumes in one folder and 12 ground truth label maps in sub folder called **labels/final** underneath it. File names and formats of volumes and labels has to match. Then, if you use the next sample button it should download automatically the 13th one (and other unsegmented ones), then you can hit the Run deep-edit\_seg button and run the model on them (inference). This is of course after you train a model.

---

<div class="post-metadata">

**Author:** ![mikebind](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@mikebind](https://discourse.slicer.org/u/mikebind)\
**Post date:** [May 26, 2022, 5:35pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/48 "2022-05-26T17:35:22Z")

</div>

Please do keep this public. I plan to try out MonaiLabel through Slicer in the next few weeks, and am trying to keep up with this discussion which I think will be very helpful when I get to starting!

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 26, 2022, 6:30pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/49 "2022-05-26T18:30:13Z")

</div>

> [@mikebind](#):
>
> Please do keep this public

Ok guys, then this may also be a timesaver when you try to set up the monailabel server with a more recent NVIDIA graphics card (NV is mandatory for CUDA):

First and most importantly, one has to upgrade to Windows 11 to run WSL2 and CUDA with recent cards.  
Still ran into

```auto
GeForce RTX 3070 Ti with CUDA capability sm_86 is not compatible with the current PyTorch installation. The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75

```

shown in the monailabel server output and training sessions failed.  
torch.version.cuda showed:

```auto
>>> import torch
>>> torch.version.cuda
'10.2'

```

So it appeared that two CUDA versions were on that computer simultaneously.  
Got this only solved by

```auto
# remove all CUDAs
sudo apt-get --purge remove "*cublas*" "cuda*" "nsight*"
sudo apt-get --purge remove "*cublas*" "*cufft*" "*curand*" 
# install torch 1.9 and CUDA 11.1
pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

```

After that

```auto
>>> import torch
>>> torch.version.cuda
'11.2'

```

and there was no more warning, monailabel training worked with that GTX 3070 Ti too.  
👾

---

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [May 26, 2022, 7:32pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/50 "2022-05-26T19:32:30Z")

</div>

I think this is a great discussion. I’d keep it public 🙂

Thanks again for sharing your experience, @muratmaga

You accurately highlighted the most important things:

1/ labels can be created/modified in any program and then submitted to MONAI Label server,  
2/ each structure should be consistently labeled with an identical label number, and  
3/ data organization in MONAI Label matters (i.e. when starting with already created labels, both images and labels should have the same name …)

Thanks, everyone!!

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [May 27, 2022, 12:20am UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/51 "2022-05-27T00:20:05Z")

</div>

> [@rbumm](#):
>
> shown in the monailabel server output and training sessions failed.

İn my experience, matching all the tools and base libraries and drivers is half the challenge of setting up a deep learning environment. You took an extra step in doing all this under wsl2. But sounds like you are making progress.

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 29, 2022, 11:58am UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/52 "2022-05-29T11:58:05Z")

</div>

Would this part of the configuration be right for setting up a lung deepedit conf ?  
How could we start a model from scratch when you can not download a pretrained model (see → # Download PreTrained Model section) ?

```auto
...
        # Single label
        #self.labels = {
        # "spleen": 1,
        # "background": 0,
        #}
        # Single label
        self.labels = {
            "right lung": 1,
            "left lung": 2,
            "airways": 3,
            "background": 0,
        }

        # Number of input channels - 4 for BRATS and 1 for spleen
        self.number_intensity_ch = 4

        network = self.conf.get("network", "dynunet")

        # Model Files
        self.path = [
            os.path.join(self.model_dir, f"pretrained_{self.name}_{network}.pt"), # pretrained
            os.path.join(self.model_dir, f"{self.name}_{network}.pt"), # published
        ]

        # Download PreTrained Model
        if strtobool(self.conf.get("use_pretrained_model", "true")):
            url = f"{self.PRE_TRAINED_PATH}/deepedit_{network}_singlelabel.pt"
            download_file(url, self.path[0])
...
``
```

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 29, 2022, 12:04pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/53 "2022-05-29T12:04:14Z")

</div>

Assuming we load a “Next sample” in 3DSlicers monailabel extension after starting the monailabel server with

```auto
monailabel start_server --app ./apps/radiology --studies ./datasets/Task06_Lung/imagesTr --conf models deepedit

```

in which standard segmentations of the above labels are created ok by monailabel:  
Does preservation of segment color matter for the learning process if you switch to an external segmentation module and the back to monailabel ?

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 29, 2022, 3:42pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/54 "2022-05-29T15:42:10Z")

</div>

> [@rbumm](#):
>
> in which standard segmentations

I ment “in which empty standard segments are created ok”

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [May 31, 2022, 1:03pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/55 "2022-05-31T13:03:16Z")

</div>

There is very very slow progress in getting a monailabel lung CT segmentation server up and running.  
What I did:

Changed segmentation.py in ./apps/radiology/lib/conf/

```auto
        # Labels
        #self.labels = {
        # "spleen": 1,
        # "right kidney": 2,
        # "left kidney": 3,
        # "gallbladder": 4,
        # "esophagus": 5,
        # "liver": 6,
        # "stomach": 7,
        # "aorta": 8,
        # "inferior vena cava": 9,
        # "portal vein and splenic vein": 10,
        # "pancreas": 11,
        # "right adrenal gland": 12,
        # "left adrenal gland": 13,
        #}

        # Labels
        self.labels = {
            "right lung": 1,
            "left lung": 2,
            "airways": 3,
        }

```

Then

```auto
monailabel start_server --app ./apps/radiology --studies ./datasets/Task06_Lung/imagesTr --conf models segmentation

```

resulted in multiple error messages and it took ages to find out that this was caused by monailabel using the multilabel pretrained\_segmentation.pt that obviously has much more labels than the three I need for lung segmentation and thus - fails.

It turned out that changing the server startup code to

```auto
monailabel start_server --app ./apps/radiology --studies ./datasets/Task06_Lung/imagesTr --conf models segmentation --conf use_pretrained_model false

```

avoids the errors and enables training.

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [May 31, 2022, 2:29pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/56 "2022-05-31T14:29:23Z")

</div>

> [@rbumm](#):
>
> avoids the errors and enables training.

Alternatively, you can rename your app to something, create a separate folder for it and pass that to the monailabel start. İ found that approach i bit cleaner when i was experimenting.

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [May 31, 2022, 5:21pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/57 "2022-05-31T17:21:42Z")

</div>

> [@rbumm](#):
>
> Does preservation of segment color matter for the learning process if you switch to an external segmentation module and the back to monailabel ?

I actually don’t know the answer to that. I suspect it does a lot. MonaiLabel works on labelmaps, not segmentation. So the issue is how those segments are converted to labelmaps. I think use of a strict terminology is necessary to for correct conversion. I am trying to learn terminology myself as you can see [here](https://discourse.slicer.org/t/tutorial-on-terminologies/23400/3)

---

<div class="post-metadata">

**Author:** ![mikebind](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@mikebind](https://discourse.slicer.org/u/mikebind)\
**Post date:** [June 2, 2022, 8:42pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/58 "2022-06-02T20:42:11Z")

</div>

@rbumm, was there a reason you chose to install monailabel in WSL2 rather than in Windows?

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [June 3, 2022, 8:13am UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/59 "2022-06-03T08:13:19Z")

</div>

This is a very good question @mikebind, for some reason and experience, I thought that this kind of server can only be set up on a Linux (sub)system, which is probably not the case. Generally not good news, because I spent days adjusting CUDA in WSL2 (see above).

The monailabel server (Windows 11, WSL2) now runs without error messages here, I can train it, but I do not get usable segmentation results in 3D Slicer yet and I decided to put this on hold until the Monailabel workshop in June or until there are new bright ideas.

Are you using monailabel directly in Windows from the powershell?

---

<div class="post-metadata">

**Author:** ![jamesobutler](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jamesobutler/32/7511_2.png) [@jamesobutler](https://discourse.slicer.org/u/jamesobutler)\
**Post date:** [June 3, 2022, 11:29am UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/60 "2022-06-03T11:29:50Z")

</div>

Yes MonaiLabel is supported on Windows. [GitHub - Project-MONAI/MONAILabel: MONAI Label is an intelligent open source image labeling and learning tool.](https://github.com/Project-MONAI/MONAILabel#installation)

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [June 3, 2022, 7:26pm UTC](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063/61 "2022-06-03T19:26:04Z")

</div>

It can confirm that actually works in Windows 11 native too.

Needed to install Python 3.9, then

```auto
git clone https://github.com/Project-MONAI/MONAILabel
$Env:PATH += ";C:\Users\rudol\MONAILabel\monailabel\scripts"
monailabel apps --download --name radiology --output apps
# modify segmentation.py by hand, add the lung and airway labels
monailabel datasets --download --name Task06_Lung --output datasets
monailabel start_server --app ./apps/radiology --studies ./datasets/Task06_Lung/imagesTr --conf models segmentation --conf use_pretrained_model false

```

I train 7-8 datasets, but all I get from auto segmentation is something like

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/8/2/8225eefd7f4dda4c4d33f20e6047b19ea4d6ef92.jpeg)

where only the first segment carries the displayed data.

[Previous page](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063.md?page=2)

[Next page](https://discourse.slicer.org/t/how-to-start-with-monailabel-for-new-models/21063.md?page=4)