Convert Monai bundle to torch file for SlicerMONAIAuto3DSeg

Support
, ,
1

Hi,

I am trying to use my network trained using Monai Auto3Dseg in the SlicerMONAIAuto3DSeg plugin. SlicerMONAIAuto3DSeg imports the torch file with a list including config information and the network weights. Monai outputs the project directory after the training, which is called the Monai bundle. Where can I find a script that converts the Monai bundle to a torch file to import in SlicerMONAIAuto3DSeg? Or do I have to write my converter?

Thanks,
Slava

1 Like
2

I think there is current work towards making this easier to do. @diazandr3s may be able to provide more details.

1 Like
3

Hi @slava,

Great question!

If you have already trained Auto3DSeg and it is for a single modality/sequence, using the model in SlicerMONAIAuto3DSeg is straightforward - you don’t need to convert anything.

Can you please provide more details of the network architecture you used? Is it SegResNet? Which image modality and region does it segment?

Just two steps: 1/ take the model.pt and 2/ create the labels.csv file.

If yes, you need to take the model.pt from the segresnet_0 folder and create the labels.csv file as I show here in these two videos:

  • Taking the model.pt:

  • Add the labels.csv file to the custom model:

To create the labels.csv file, you may find this list useful: SlicerTotalSegmentator/TotalSegmentator/Resources/totalsegmentator_snomed_mapping.csv at main · lassoan/SlicerTotalSegmentator · GitHub

  • Modify the models.json in the SlicerMONAIAuto3DSeg and place the model in the cache folder:

Hope this helps,

Please let us know.

4

Hi Andres,

Thank you for jumping in. I trained DiNTS so far. The problem is that the .pt file Monai output does not have a config key inside, which triggers the error below. The config key the plugin tries to read has all the information about the data’s pre- and post-processing. I believe Monai changed the structure of the .pt file since they use bundles, and all pre- and post-processing information is in the configs folder.

Thanks,
Slava

if ‘config’ not in checkpoint:
raise ValueError(‘Config not found in checkpoint (not a auto3dseg/segresnet model):’+str(model_file))

5

Hi Andres,

I want to confirm that Moani outputs strictly network weights into a torch file. The torch files provided with the plugin have the structure dict_keys([‘state_dict’, ‘config’, ‘epoch’, ‘best_metric’]), where ‘state_dict’ is the weights and ‘config’ has all information about roi_size, etc.

Thanks,
Slava

6

It is easy to change the plugin code to make it compatible with the Monai Bandle; you just need to parse YAML files in the config folder and read network weight from torch file. Please let me know if you plan to do it in the next few days. I can do it myself, but I don’t want to duplicate the effort if you plan to do it.

1 Like
7

Hi @slava,

Thanks for the update.

Would it be possible for you to train the Auto3DSeg using SegResNet instead of DiNTS?

It should be something like this:

python -m monai.apps.auto3dseg AutoRunner run --input ./configs/input_task.yaml --algos segresnet --work_dir ./output_folder

Then it’ll be straightforward to use on this module.

I know there are still some discrepancies between the Bundle created by MONAI Auto3DSeg and the one available in the model zoo. But you don’t need to create any bundle to use the SlicerMONAIAuto3DSeg

A further analysis should be done before starting any PR. I’d strongly suggest you comment on your experience here: Using one Bundle generated model in Auto3DSeg in MONAI Label · Project-MONAI/MONAILabel · Discussion #1591 · GitHub

or here: Unification of auto3dseg algorithm template format and MONAI bundle format · Issue #7467 · Project-MONAI/MONAI · GitHub

I hope this helps,

8

@diazandr3s This is very helpful! Thank you.

I’m new to Auto3DSeg and was wondering if it’s possible to get a .pt file of the ensemble itself. I could only find the .pt files for the individual models in the bundle generated.

1 Like
9

Hi @nmod,
As far as I am aware, each Pytorch neural network is saved as its own checkpoint (.pt) file. Therefore, since the Auto3DSeg ensemble is not a single neural network, the ensemble results cannot be used directly in the Slicer extension without software coding changes. The SlicerMONAIAuto3DSeg extension code uses only the SegResNet model, so you could follow @Diaz-Pinto, Andres previous advice and use the .pt file corresponding to the trained SegResNet model, but not the entire ensemble. Or you could retrain with only the segresnet model and use that .pt as Andres also mentioned. I hope this helps.

Curt

1 Like
10

Thank you so much! This is very helpful

1 Like