Apart from supporting the
MONAILabel team, I would like to work on more generic and lower-level compatibility issues between PyTorch and Slicer.
Basically, I imagine the following scenario: a user has trained a deep learning segmentation model using PyTorch (and possibly TorchIO, MONAI or both). They want users (e.g., clinicians) to be able to use the model on their own data, without the need to code. The best solution is probably to contribute an extension. (I am in this situation, with
resseg and its corresponding extension
Three issues I would like to address:
- How to install PyTorch inside Slicer. The main question is whether to install a version with GPU support and, if it does, which version of the CUDA toolkit to install. I did a bit of work on this during the development of the
- How to handle the necessary conversion of Slicer nodes (e.g.
vtkMRMLScalarNode) to PyTorch objects (e.g.
torch.Tensor). A few additions to
slicer.util might help here.
- Possibly, contributing a full tutorial with a toy example using a publicly available dataset such as TorchIO’s
IXITiny or a dataset from the Medical Segmentation Decathlon*
If someone is interested in this stuff, please let me know and let’s work together!
Some related projects that are probably worth looking at are DeepInfer and TOMAAT.
*Many images from the Medical Decathlon cannot be easily read by Slicer due to their 4D shape. This can maybe be addressed within the
MONAILabel projects – @diazandr3s, @SachidanandAlle