TensorFlow x Slicer

Dear Slicer Community,

I’m new here and I wanted to share an extension I made to use my TensorFlow models for image segmentation directly into slicer. I’ve already read here that most deep learning models are developed under Pytorch in the medical field, but I though it would be fun trying to create an extension focused on TensorFlow.

The model input size is detected to resize the selected image/volume to compatible size and range of values.

For now the segmentation can only be segmented slice by slice (1 image as input - 1 segmentation as output) but I’ll try to implement 2.5D approach (N images as input - 1 segmentation as output) and 3D approach (full volume as input - full volume as output).

Rep of the extension : https://github.com/VincentMillotMaysounabe/Slicer-TensorFlow

If any of you have comments on how I could enhance the project it could be really precious to me.

Thanks,
Vincent

1 Like

Hi - welcome, this sounds really interesting, and yes, it’s common. to use pytorch but tensorflow should work fine too if it gets installed right. For me on a linux test with a recent build, pip_install("tensorflow[and-cuda]") in the python console completed but crashed whtn I tried to import it. I suspect there may be similar issues with cuda drivers since there have been many with pytorch.

BTW, your repository would benefit with some examples and screenshots so it’s clear what the module currently does.

Also since it’s pure python you should be able to provide instructions for people to install it locally. This doesn’t need to be in the extension manager until it’s ready for use by the general user population.

Hi
Thanks so much for your answer !
I was not aware of those installation issues since it worked fine for me. I’m on Windows using Slicer 5.6.1 and TensorFlow 2.15.0. I’d like to investigate more about this, have you any idea of where to start ?

I recently updated my repository with a quick overview of the extension and a full tutorial with example files and a -not so accurate- U-net model for prostate segmentation.

Regarding GPU driver issues you can just search this discourse for questions about TotalSegmentator and you’ll find a lot of questions about installation issues. Maybe more on linux than windows but both come up. Usually they can be resolved if someone has the patients and some technical experience, but it’s a barrier to widespread use.