Hi all,
Apart from supporting the MONAILabel team, I would like to work on more generic and lower-level compatibility issues between PyTorch and Slicer.
Basically, I imagine the following scenario: a user has trained a deep learning segmentation model using PyTorch (and possibly TorchIO, MONAI or both). They want users (e.g., clinicians) to be able to use the model on their own data, without the need to code. The best solution is probably to contribute an extension. (I am in this situation, with resseg and its corresponding extension SlicerEPISURG).
Three issues I would like to address:
- How to install PyTorch inside Slicer. The main question is whether to install a version with GPU support and, if it does, which version of the CUDA toolkit to install. I did a bit of work on this during the development of the
SlicerTorchIOextension. - How to handle the necessary conversion of Slicer nodes (e.g.
vtkMRMLScalarNode) to PyTorch objects (e.g.torch.Tensor). A few additions toslicer.utilmight help here. - Possibly, contributing a full tutorial with a toy example using a publicly available dataset such as TorchIO’s
IXITinyor a dataset from the Medical Segmentation Decathlon*
If someone is interested in this stuff, please let me know and let’s work together!
Some related projects that are probably worth looking at are DeepInfer and TOMAAT.
*Many images from the Medical Decathlon cannot be easily read by Slicer due to their 4D shape. This can maybe be addressed within the MONAILabel projects – @diazandr3s, @SachidanandAlle