# Neuroimage annotation using 3D Slicer for AI training 

**URL:** <https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034>\
**Category:** Support\
**Created:** [April 12, 2021, 4:52am UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034 "2021-04-12T04:52:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dhruba](https://avatars.discourse-cdn.com/v4/letter/d/e495f1/32.png) [@Dhruba](https://discourse.slicer.org/u/Dhruba)\
**Post date:** [April 12, 2021, 4:52am UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034/1 "2021-04-12T04:52:10Z")

</div>

Operating system: Windows 10  
Slicer version: 4.11

I want to train a Neural Network to detect a vessel in CT Perfusion data. For that, I need to annotate some DICOM images using the “3D slicer”. I want the network to detect the vessel by putting a rectangular box on the region. I want to use the ‘MarkupsPlane’ to annotate the images.

Should I save the annotation and DICOM images as “.nrrd” and “.json” files to read in the neural network? I am guessing the “.json” file will work as the ground-truth label for detection.

Also, some of the patients’ data were collected with a scanner that creates 190 slices per scan and some were collected with a scanner that creates 158 slices. So, half of the “.nrrd” has a size of 512x512x190 and the rest has 512x512x158. Can I resize them to a common size (e.g. 128x128x128) before feeding them in the neural net?

---

<div class="post-metadata">

**Author:** ![Fernando](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fernando/32/5640_2.png) [@Fernando](https://discourse.slicer.org/u/Fernando)\
**Post date:** [April 12, 2021, 10:29pm UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034/2 "2021-04-12T22:29:15Z")

</div>

Hi, @Dhruba. If you save your images in NRRD, you can use [TorchIO](https://torchio.readthedocs.io/) to read them and train using patches of that common size.

You can try the transforms using the [SlicerTorchIO](https://torchio.readthedocs.io/interfaces/index.html#d-slicer-gui) extension.

---

<div class="post-metadata">

**Author:** ![Dhruba](https://avatars.discourse-cdn.com/v4/letter/d/e495f1/32.png) [@Dhruba](https://discourse.slicer.org/u/Dhruba)\
**Post date:** [April 13, 2021, 1:34am UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034/3 "2021-04-13T01:34:20Z")

</div>

Thanks @Fernando . But I want to use TensorFlow for implementing the Neural Net. In that case, what should I use? I guess I can use TorchIO only if I implement my DNN using pytorch.  
Also, is “.json” file enough for ground truth labeling?

---

<div class="post-metadata">

**Author:** ![Fernando](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fernando/32/5640_2.png) [@Fernando](https://discourse.slicer.org/u/Fernando)\
**Post date:** [April 13, 2021, 7:53am UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034/4 "2021-04-13T07:53:00Z")

</div>

Why do you want to use TensorFlow? I think it’s possible to mix TorchIO and TensorFlow, but I haven’t tried. I don’t know any currently-maintained framework for medical images with TensorFlow, we are all using PyTorch now (MONAI, TorchIO, nnU-Net, Rising, batchgenerators, medicaltorch…). Maybe you can use pymia. But this is off-topic.

I think JSON is fine, as all you need is 6 numbers representing the bounding box. You could even use a text file, a CSV, a .npy file… It doesn’t really matter, as long as you can read it during training.

---

<div class="post-metadata">

**Author:** ![Dhruba](https://avatars.discourse-cdn.com/v4/letter/d/e495f1/32.png) [@Dhruba](https://discourse.slicer.org/u/Dhruba)\
**Post date:** [April 13, 2021, 4:35pm UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034/5 "2021-04-13T16:35:05Z")

</div>

I have never used PyTorch before. I am comfortable with TensorFlow only. Maybe I will switch over to PyTorch in future.  
I have attached a pic of my “.json” file generated by 3D Slicer after using markups Plane. I am not sure which 6 numbers you meant. In my “.json”, there are 3 values in ‘position’, 9 values in ‘orientation’. Also some other numbers for ‘color’ and stuff.

 ![markups](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/6/0/60aeb3d090b3621e961a5f5ec9bcd792ef28b273.png)

---

<div class="post-metadata">

**Author:** ![Fernando](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fernando/32/5640_2.png) [@Fernando](https://discourse.slicer.org/u/Fernando)\
**Post date:** [April 13, 2021, 4:59pm UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034/6 "2021-04-13T16:59:15Z")

</div>

I am not familiar with the new Markups system, but that seems to be the contents of a `.mrk.json` file used to store fiducial points in a markups node. Basically, each 3D point can be represented with 3 coordinates. You mentioned a “rectangular box”, so I assumed you wanted to save an .acsv file representing an ROI node, which is a 3D bounding box that can be represented using 6 numbers, e.g., the center and the axes lengths.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 14, 2021, 4:20am UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034/7 "2021-04-14T04:20:48Z")

</div>

If you fully define a plane then you will have 3 control points. First control point position is the center, second control points position determines is the x axis direction and x size, third control point determines rotation around x axis and y size of the plane.

You might find the new Markups ROI easier to use. The ROI is axis-aligned by default and specified by center, orientation, and size fields.

---

<div class="post-metadata">

**Author:** ![Dhruba](https://avatars.discourse-cdn.com/v4/letter/d/e495f1/32.png) [@Dhruba](https://discourse.slicer.org/u/Dhruba)\
**Post date:** [April 14, 2021, 5:07am UTC](https://discourse.slicer.org/t/neuroimage-annotation-using-3d-slicer-for-ai-training/17034/8 "2021-04-14T05:07:17Z")

</div>

Thanks @lassoan . I am going to use the ROI. I think it is better suited and easier to understand. I have attached the “.acsv” file generated for ROI bounding box. It has 6 values. I think the top 3 points are the center coordinates for x, y , and z. The bottom 3 points represent the length of the box along the x, y, and z-axis. Is this right?

 ![12](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/f/1/f12558b50a272d359a5c90e8fe45056f8b8488f1.png)