Slicer interact with external deep learning segmentation software

Development
1

Hi,

I am new to the slicer. I would like to incorporate slicer with our own segmentation module. Since our module is doing segmentation step-by-step (e.g., vertebrae by vertebrae), so we would like to show the user partial segmentation and they can help us improve the partial segmentation. Specificlly, the workflow is:

  1. Slicer sends the scans to the program
  2. program computes the segmentation of the first part and sends it back to Slicer
  3. Slicer shows this part to the user and the user refines it. Then Slicer sends refined segmentation to the program.
  4. Program computes the segmentation of the next part and repeats steps 2-4.

I am wondering what kind of stuff should we use for achieving the goal (maybe writing a module)? And also I am wondering is there something similar in the community that I can refer to?

Thanks!

2

There may be many approaches.

  • You can use OpenIGTLink to send/receive image data
  • You can use web sockets if for some reason the above is not possible
  • Migrate the auto-segmentation step to Slicer

For more concrete suggestions more concrete information would be useful.

3

Hi Csaba,

Thanks for your reply! I want to migrate the auto-segmentation step to Slicer. I am looking for some suggestion or some similar extensions of Slicer to start.

4

If your auto-segmentation is deep learning based, then currently the most future-proof choice in Slicer would be to use MONAI. The project week will take place in two weeks (a free online event) where you can get up-to-date information about that. I suggest joining for the MONAI day on Tuesday June 29.

5

Hi Csaba,

Thanks for the reply and pointing the MONAI. But I am not looking for a framework for deep learning, I am quite familiar with PyTorch but not the Slicer. So I am looking for some Slicer modules that are designed to interact with some custom modules.

6

MONAILabel is developed exactly for solving your use case. You don’t need to use MONAI at all (or even pytorch), you can use your own network. You can of course redevelop and maintain a similar feature independently, but you may save time overall if you use/extend/customize a module developed by others. Even if you end up implementing your own solution, you could learn a lot from it as an example.

1 Like
7

Hi Andras,

This project seems perfectly solve my question. Thanks!