Neuroimage annotation using 3D Slicer for AI training

Operating system: Windows 10
Slicer version: 4.11

I want to train a Neural Network to detect a vessel in CT Perfusion data. For that, I need to annotate some DICOM images using the “3D slicer”. I want the network to detect the vessel by putting a rectangular box on the region. I want to use the ‘MarkupsPlane’ to annotate the images.

Should I save the annotation and DICOM images as “.nrrd” and “.json” files to read in the neural network? I am guessing the “.json” file will work as the ground-truth label for detection.

Also, some of the patients’ data were collected with a scanner that creates 190 slices per scan and some were collected with a scanner that creates 158 slices. So, half of the “.nrrd” has a size of 512x512x190 and the rest has 512x512x158. Can I resize them to a common size (e.g. 128x128x128) before feeding them in the neural net?

Hi, @Dhruba. If you save your images in NRRD, you can use TorchIO to read them and train using patches of that common size.

You can try the transforms using the SlicerTorchIO extension.

Thanks @Fernando . But I want to use TensorFlow for implementing the Neural Net. In that case, what should I use? I guess I can use TorchIO only if I implement my DNN using pytorch.
Also, is “.json” file enough for ground truth labeling?

Why do you want to use TensorFlow? I think it’s possible to mix TorchIO and TensorFlow, but I haven’t tried. I don’t know any currently-maintained framework for medical images with TensorFlow, we are all using PyTorch now (MONAI, TorchIO, nnU-Net, Rising, batchgenerators, medicaltorch…). Maybe you can use pymia. But this is off-topic.

I think JSON is fine, as all you need is 6 numbers representing the bounding box. You could even use a text file, a CSV, a .npy file… It doesn’t really matter, as long as you can read it during training.

1 Like

I have never used PyTorch before. I am comfortable with TensorFlow only. Maybe I will switch over to PyTorch in future.
I have attached a pic of my “.json” file generated by 3D Slicer after using markups Plane. I am not sure which 6 numbers you meant. In my “.json”, there are 3 values in ‘position’, 9 values in ‘orientation’. Also some other numbers for ‘color’ and stuff.

I am not familiar with the new Markups system, but that seems to be the contents of a .mrk.json file used to store fiducial points in a markups node. Basically, each 3D point can be represented with 3 coordinates. You mentioned a “rectangular box”, so I assumed you wanted to save an .acsv file representing an ROI node, which is a 3D bounding box that can be represented using 6 numbers, e.g., the center and the axes lengths.

If you fully define a plane then you will have 3 control points. First control point position is the center, second control points position determines is the x axis direction and x size, third control point determines rotation around x axis and y size of the plane.

You might find the new Markups ROI easier to use. The ROI is axis-aligned by default and specified by center, orientation, and size fields.

1 Like

Thanks @lassoan . I am going to use the ROI. I think it is better suited and easier to understand. I have attached the “.acsv” file generated for ROI bounding box. It has 6 values. I think the top 3 points are the center coordinates for x, y , and z. The bottom 3 points represent the length of the box along the x, y, and z-axis. Is this right?