Hello
I am currently using MONAILabel extension with Slicer and I would like to ask how I can define which data is assigned for the calculation of the validation Dice similarity metric. I know I can adjust the percentage which is used for this calculation but can I also select on which data would this calculation based on?
Thank you 
Good question. Here is the method used to split the training and validation dataset: https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic_train.py#L394-L412
you could replace this method in the training file (BasicTrainTask class) like this: https://github.com/Project-MONAI/MONAILabel/blob/testsMICCAI/sample-apps/deepedit_multilabel/lib/train.py#L242-L256
hope this helps,
Hello again,
Little bit late but thank you for your response
Truth to be told, I made this question because I knew in advance I will have an issue there. Well the time has come to resolve my question.
However, the last time maybe I did not explain what I did. FIrst, I used MONAI app in Slicer. I used the radiology app and trained a model from scratch . I run the MONAI in a server with this command :
As you can see I have a specific folder with the training volumes. I know the 20% of this dataset is used for validation. However, I cannot see when I call this command where exactly the validation dataset is defined.
I saw your answer, but I cannot find the script (train.py) which I must modify and when I start the server (with the command I provided) to use the specified validation dataset.
I know, it is a kind of stupid question but a I am a beginner here:)
Thank you and sorry for the long text.
I see you are using the segmentation model. Here you can find the trainer file for this model: https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/trainers/segmentation.py
As you can see, the segmentation train class inherits the interface BasicTrainTask: https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/trainers/segmentation.py#L41
In that class is where the training validation split happens: https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic_train.py#L394
Hope this helps,
Hello and thank you for your answers
I will check it and allow me to answer back if I cannot resolve it.
Thank you for the continued support
Best
Spiros
Hello again
Okay, as you can see in the image, I did find the trainer file for the model
Another, ‘‘stupid’’ or easy question : How I can access the script basic_train.py file to change the val_datalist ?
I saw from your last image that this file is the folder named tasks.
However, the files inside the radiology app that I can see are those:
As I saw from your last image this class is located is inside a folder named Tasks, however, I cannot find it.
I know, these are easy questions but forgive me in advance, I am not an expert on these.
For some reason, I just saw this reply.
You don’t need to modify the basic_train file. You just modify the “partition_datalist” method in the trainer class.
When you inherit a class, all methods defined there overwrite the ones from the father class.
Hope this makes sense and sorry for the late reply.
Hey and thank you for your answer. I understand what you mean, however, still I can not find the partition_datalist method . I open up the segmentation.py file and I can see the BasicTrainTask.
But how can I access the class and change the partition_list.
In general, I have trained a couple of models using MONAILabel using MR images I provided without changing the partition list etc.
I want to know which images are used automatically by MONAI to calculate the validation Dice similarity metric. Are they picked randomly ? Are the fisrt 20%?
Thank you again and I know, my question might seem simple
The partition_datalist method is not defined in the segmentation.py file. It is part of the BasicTrainTask class (parent class). If you want to see that method in the segmentation.py file, you have to write it there and modify it as you wish.
This is the default way MONAI Label splits the validation and training set: MONAILabel/monailabel/tasks/train/basic_train.py at main · Project-MONAI/MONAILabel · GitHub
Hope this helps,
Question @diazandr3s ,
Is ther any way to make a partition_datalist method to give the validation dataset path throw the init server command as we usially do with training dataset path (–studies)?
It should be nice to use the same trainer with multiple models…
Thanks in advance!
Hi @apparrilla,
That’s a good idea.
You could add an argument to specify the validation path here: MONAILabel/monailabel/config.py at a5733ebcf113b6811a85eeaf75af0f9e929c3567 · Project-MONAI/MONAILabel · GitHub
and in the main class here: MONAILabel/monailabel/main.py at a5733ebcf113b6811a85eeaf75af0f9e929c3567 · Project-MONAI/MONAILabel · GitHub
Then you should be able to use this argument in any MONAI Label app as it uses this main class: MONAILabel/monailabel/interfaces/app.py at a5733ebcf113b6811a85eeaf75af0f9e929c3567 · Project-MONAI/MONAILabel · GitHub
Hope this helps,
