# Validation\_set\_Accuracy in MONAILabel

**URL:** <https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027>\
**Category:** Support\
**Tags:** segmentation, monailabel\
**Created:** [August 7, 2023, 2:05pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027 "2023-08-07T14:05:10Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Spiros\_Karkavitsas](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/spiros_karkavitsas/32/18224_2.png) [@Spiros\_Karkavitsas](https://discourse.slicer.org/u/Spiros_Karkavitsas)\
**Post date:** [August 7, 2023, 2:05pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/1 "2023-08-07T14:05:10Z")

</div>

Hello

I am currently using MONAILabel extension with Slicer and I would like to ask how I can define which data is assigned for the calculation of the validation Dice similarity metric. I know I can adjust the percentage which is used for this calculation but can I also select on which data would this calculation based on?

Thank you 🙂

---

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [August 8, 2023, 8:43pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/2 "2023-08-08T20:43:48Z")

</div>

Hi @Spiros_Karkavitsas,

Good question. Here is the method used to split the training and validation dataset: [https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic\_train.py#L394-L412](https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic_train.py#L394-L412)

you could replace this method in the training file (BasicTrainTask class) like this: [https://github.com/Project-MONAI/MONAILabel/blob/testsMICCAI/sample-apps/deepedit\_multilabel/lib/train.py#L242-L256](https://github.com/Project-MONAI/MONAILabel/blob/testsMICCAI/sample-apps/deepedit_multilabel/lib/train.py#L242-L256)

hope this helps,

---

<div class="post-metadata">

**Author:** ![Spiros\_Karkavitsas](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/spiros_karkavitsas/32/18224_2.png) [@Spiros\_Karkavitsas](https://discourse.slicer.org/u/Spiros_Karkavitsas)\
**Post date:** [December 1, 2023, 11:38am UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/3 "2023-12-01T11:38:00Z")

</div>

Hello again,

Little bit late but thank you for your response 🙂

Truth to be told, I made this question because I knew in advance I will have an issue there. Well the time has come to resolve my question.

However, the last time maybe I did not explain what I did. FIrst, I used MONAI app in Slicer. I used the radiology app and trained a model from scratch . I run the MONAI in a server with this command :

 ![MONAIquestion](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/d/9/d93b68c4a408f28bf02b0d0a5a4469ef68b0dd19.png)

As you can see I have a specific folder with the training volumes. I know the 20% of this dataset is used for validation. However, I cannot see when I call this command where exactly the validation dataset is defined.

I saw your answer, but I cannot find the script (train.py) which I must modify and when I start the server (with the command I provided) to use the specified validation dataset.

I know, it is a kind of stupid question but a I am a beginner here:)

Thank you and sorry for the long text.

---

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [December 5, 2023, 10:43am UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/4 "2023-12-05T10:43:16Z")

</div>

Hi @Spiros_Karkavitsas,

I see you are using the segmentation model. Here you can find the trainer file for this model: [https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/trainers/segmentation.py](https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/trainers/segmentation.py)

As you can see, the segmentation train class inherits the interface **BasicTrainTask** : [https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/trainers/segmentation.py#L41](https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/trainers/segmentation.py#L41)

In that class is where the training validation split happens: [https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic\_train.py#L394](https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic_train.py#L394)

Hope this helps,

---

<div class="post-metadata">

**Author:** ![Spiros\_Karkavitsas](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/spiros_karkavitsas/32/18224_2.png) [@Spiros\_Karkavitsas](https://discourse.slicer.org/u/Spiros_Karkavitsas)\
**Post date:** [December 5, 2023, 1:35pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/5 "2023-12-05T13:35:51Z")

</div>

Hello and thank you for your answers

I will check it and allow me to answer back if I cannot resolve it.

Thank you for the continued support 🙂

Best  
Spiros

---

<div class="post-metadata">

**Author:** ![Spiros\_Karkavitsas](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/spiros_karkavitsas/32/18224_2.png) [@Spiros\_Karkavitsas](https://discourse.slicer.org/u/Spiros_Karkavitsas)\
**Post date:** [December 6, 2023, 12:11pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/6 "2023-12-06T12:11:07Z")

</div>

Hello again

Okay, as you can see in the image, I did find the trainer file for the model

 ![MONAIquestion](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/8/c/8c81997fcc6d72cfc63e0dba3e0a22728a48724b.png)

Another, ‘‘stupid’’ or easy question 🙂 : How I can access the script basic\_train.py file to change the val\_datalist ?

I saw from your last image that this file is the folder named tasks.

However, the files inside the radiology app that I can see are those:

 ![MONAIquestion](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/5/e/5e044e1710de774f030b45c4b7b4f7bf5240950d.png)

As I saw from your last image this class is located is inside a folder named Tasks, however, I cannot find it.

I know, these are easy questions but forgive me in advance, I am not an expert on these.

---

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [February 20, 2024, 2:20pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/7 "2024-02-20T14:20:17Z")

</div>

Hi @Spiros_Karkavitsas,

For some reason, I just saw this reply.

You don’t need to modify the basic\_train file. You just modify the “[partition\_datalist](https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic_train.py#L394)” method in the trainer class.  
When you inherit a class, all methods defined there overwrite the ones from the father class.

Hope this makes sense and sorry for the late reply.

---

<div class="post-metadata">

**Author:** ![Spiros\_Karkavitsas](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/spiros_karkavitsas/32/18224_2.png) [@Spiros\_Karkavitsas](https://discourse.slicer.org/u/Spiros_Karkavitsas)\
**Post date:** [March 1, 2024, 6:13pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/8 "2024-03-01T18:13:02Z")

</div>

Hey and thank you for your answer. I understand what you mean, however, still I can not find the partition\_datalist method . I open up the segmentation.py file and I can see the BasicTrainTask.

But how can I access the class and change the partition\_list.

In general, I have trained a couple of models using MONAILabel using MR images I provided without changing the partition list etc.  
I want to know which images are used automatically by MONAI to calculate the validation Dice similarity metric. Are they picked randomly ? Are the fisrt 20%?

Thank you again and I know, my question might seem simple 🙂

---

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [March 2, 2024, 10:33am UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/9 "2024-03-02T10:33:47Z")

</div>

Hi @Spiros_Karkavitsas,

The _partition\_datalist_ method is not defined in the segmentation.py file. It is part of the BasicTrainTask class (parent class). If you want to see that method in the segmentation.py file, you have to write it there and modify it as you wish.

This is the default way MONAI Label splits the validation and training set: [MONAILabel/monailabel/tasks/train/basic\_train.py at main · Project-MONAI/MONAILabel · GitHub](https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic_train.py#L394-L412)

Hope this helps,

---

<div class="post-metadata">

**Author:** ![apparrilla](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/apparrilla/32/9230_2.png) [@apparrilla](https://discourse.slicer.org/u/apparrilla)\
**Post date:** [November 6, 2024, 9:09pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/10 "2024-11-06T21:09:46Z")

</div>

Question @diazandr3s ,  
Is ther any way to make a partition\_datalist method to give the validation dataset path throw the init server command as we usially do with training dataset path (–studies)?  
It should be nice to use the same trainer with multiple models…

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![diazandr3s](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/diazandr3s/32/9973_2.png) [@diazandr3s](https://discourse.slicer.org/u/diazandr3s)\
**Post date:** [November 14, 2024, 5:15pm UTC](https://discourse.slicer.org/t/validation-set-accuracy-in-monailabel/31027/11 "2024-11-14T17:15:07Z")

</div>

Hi @apparrilla,

That’s a good idea.

You could add an argument to specify the validation path here: [MONAILabel/monailabel/config.py at a5733ebcf113b6811a85eeaf75af0f9e929c3567 · Project-MONAI/MONAILabel · GitHub](https://github.com/Project-MONAI/MONAILabel/blob/a5733ebcf113b6811a85eeaf75af0f9e929c3567/monailabel/config.py#L28)

and in the main class here: [MONAILabel/monailabel/main.py at a5733ebcf113b6811a85eeaf75af0f9e929c3567 · Project-MONAI/MONAILabel · GitHub](https://github.com/Project-MONAI/MONAILabel/blob/a5733ebcf113b6811a85eeaf75af0f9e929c3567/monailabel/main.py#L38)

Then you should be able to use this argument in any MONAI Label app as it uses this main class: [MONAILabel/monailabel/interfaces/app.py at a5733ebcf113b6811a85eeaf75af0f9e929c3567 · Project-MONAI/MONAILabel · GitHub](https://github.com/Project-MONAI/MONAILabel/blob/a5733ebcf113b6811a85eeaf75af0f9e929c3567/monailabel/interfaces/app.py#L75)

Hope this helps,