Develop a slicer extension with Deep Learning

Support
1

Operating system: win 10
Slicer version:4.11.0
Expected behavior:
Actual behavior:

Hi, everyone. I am doing some investigations to know about if it is possible to develop a slicer extension with Deep Learning on the current version slicer 4.11.0.

Also，Deep Learning Framework, such as tensorflow, Keras and Pytorch. Which one is better or more compatiable for a slicer extension development ?

Any references or examples are welcome.

Any help would be appreciated.

2

Hi,

We made a tutorial that guides you through how to train a deep learning model with data exported from Slicer, and how to use the trained model in a Slicer module for predictions:
http://www.slicerigt.org/wp/cars-2019-tutorial/
This example uses webcam images, but you can use 99% of the examples for any medical image type.

You will find some example code in this project too, mostly on ultrasound segmentation:

Slicer can use TensorFlow, so I suggest you use Keras (now officially part of TensorFlow). I couldn’t figure out how to use PyTorch in Slicer, but it must be possible somehow. Keras seems to be more popular anyway.

I hope this helps,
Tamas

6 Likes
3

Thanks @ungi for the great resources! I will check them out too.

I would just like to add that I use PyTorch from Slicer sometimes, so I can explain how I have gotten it to work…

If you go to pytorch.org, it will provide the proper install command for your configuration.
Then you can install PyTorch in Slicer’s Python via:

C:\path\to\Slicer\bin\PythonSlicer -m <install-command>

For example, I use Windows 10 and do not have GPU, so I used the command:

C:\path\to\Slicer\bin\PythonSlicer -m pip install torch==1.3.1+cpu torchvision==0.4.2+cpu -f https://download.pytorch.org/whl/torch_stable.html

Once installed, you can import PyTorch into Slicer’s Python interactor with the standard command:

import torch
import torchvision
5 Likes