I am working on a project where I generate the segmentation of blood vessels in the brain using some other software and compute their centrelines using the SlicerVMTK extension. However, a common problem with the segmentation is that the same vessel may get segmented into multiple disconnected segments. I cannot come up with a very good way to connect these fragmented segments together automatically, so I wanted to implement a module/extension on 3D slicer where the user can click two segments and then run some algorithms to connect these two segments or just connect their centerlines. I am labelling disconnected islands in a segmentation using different integers first, so that when I load it into 3D slicer, different islands will have different colors. I want to ask how I can make it interactive so that the user can toggle-select two islands? I want some effect that is similar to selecting the endpoints/fiducial points in 3D slicer.
I assume you are referring to arteries of the circle of Willis from CT scans.
Have you tried segmenting the same structures with Slicer? Usually āGrow from seedsā effect of the āSegment editorā gives nice results. You may ease the process by extracting the brain first (TotalSegmentator, Swiss skull stripper, HD-BETā¦), apply āSplit volumeā or āMask volumeā and increase the resolution of the resulting volume with āResample scalar volumeā. āGrow from seedsā should then play nicely. The quality of the master input is always of great import.
If you are working with MRI, āGrow from seedsā still performs well.
Iām suggesting here a simple pre-processing to avoid a complex post-processing.
Thank you for the suggestions. I am actually working with MR angiography data and we are looking at very small brain vessels, which is a much more challenging task than segmenting the large arteries. We are using a deep learning model for segmentation, which does a good job, but the same vessel still often get segmented into disconnected parts. This is also due to the poor signal from the small vessels themselves. Thus, we are hoping to add some user-correction in the post-processing stage.
You can create a python module which creates a markups node and observes it. When the user places a point of that node by clicking, you can identify the closest vessel segment, and use this point placement as your āselectionā mechanism. When two different segments have been selected by this method, you could connect the closest ends of the centerlines of those two to create a new, combined centerline. To add a simple segmentation connecting the two ends, you could use the Draw Tube effect from SegmentEditorExtraEffects extension to make a cylindrical linkage. The radius for that cylinder could be based on the vessel radius at the two ends from VMTK. Then you could merge the original two vessel segments and the linkage segment to make one unified vessel segment. The old segments could be discarded or hidden and the selection markups control points could be discarded, and you would be ready to select more segments to join.
Thank you very much for your kind suggestions Mike! What you described is exactly what I need. However, although I am proficient in python, I am very new to coding with 3D Slicer, and I have just started learning how to create a new module and the concepts of the widget and logic functions. I wonder if you mind giving me some hint on how to achieve those steps in your suggestion and possibly some code example? For instance, should I code a logic function that creates a markup node? How to āobserveā it? How can I make a function to wait a user to select two segments and then execute something else?
Sorry if these questions are two basic or broad, but I am really struggling to get my head around how the module code works and to find the right functions from slicer to call, so I will really appreciate some help with this.
The details of how to achieve this are too big for a quick response here, but Iāll try to get you pointed in some helpful directions and you can always come back and ask for help when you are getting stuck.
At a high level, the idea behind dividing the widget and logic into separate classes is that the widget manages all interaction with GUI elements, while the logic carries out all the functionality (the working-with and modifying data stuff) you want your module to have. The major benefit of this division is that outside code (i.e. modules that other people write) can use the functionality you provide without having to instantiate your widget GUI. Maybe this will be an easy to understand concept for you since it sounds like you have a solid programming background, but this was something I struggled with at first (deciding how functions should be structured). The good news is that, as you are learning, nothing bad happens if you put functionality in the widget class that properly belongs in the logic class. And, as you get used to it, it will steadily become clearer what belongs where.
Some recommendations, steps to take, and resources:
Enable developer mode in your Slicer settings (Edit menu ā Application Settings ā Developer ā Enable developer mode checkbox). This will show some helpful extra buttons on all python scripted modules which will be very helpful while developing your module. The āReloadā button reloads the module from the source code, which is very helpful when you are actively making and testing changes (usually thereās no need to restart Slicer to test changes). The āEditā button opens the module source code in your default editor. This can be very useful for looking at other modulesā code as well as your own. The āEdit UIā button launches Qt Designer and allows you to edit the GUI elements of your module in a reasonably intuitive way.
I highly recommend using Extension Wizard module and starting from there with your first module. The template code there is a simple working module which allows thresholding an input volume. As a minimal first example, I would recommend trying to add a new button to this module and make it do something like add one to all voxel values of the input. In order to do this, youāll need to look closely at the template code and try to mimic what is done for the existing Apply button. Youāll need to create the button in Qt Designer, connect the button click signal to a callback in your widgetās setup function, write the callback function to gather any needed inputs, pass those inputs to a logic function which does the actual work of taking the input volume node, getting the voxel values, adding one, and either updating the original input volume or creating a new output volume with the new values. This should be a very educational exercise, and by the time youāve accomplished it, youāll have a better sense of how Slicer modules typically work and pass around data.
Next, start thinking about what inputs your functions will need. You will need a selector to choose the segmentation node which contains all your vessel segments, and you will also need the ability to place markups points. For the first, in Qt Designer, you want to add a qMRMLNodeComboBox and set the nodeTypes it allows you to select to be āvtkMRMLSegmentationNodeā. For the second, a handy widget is qSlicerMarkupsPlaceWidget. Lastly, you probably want a button which you click once youāve identified the two segments you want to link and which triggers carrying out the actual linkage.
Thatās probably all you really need on the GUI side. When you click the button, the selected segmentation node and markups node should be passed to a logic function which is supposed to carry out all of the work. You can start off with a stub function there which just lets you know it has been called by printing something out (print statements or logging statements show up in the python interactor in Slicer as well as in the Error Log window (Ctrl-0 or the X in the lower right of the status bar)).
The rest of the work is actually implementing what you want to happen, given the inputs. The good news is that this is just pure python coding Here are the pieces I see that you will need:
A function finding the closest segment centerline endpoint to a markups point. Iām not sure what format your centerline data is in after your initial processing, but at minimum, you will need some way of keeping track of which segment goes with which centerline. To find the closest centerline endpoint to your first clicked point, I would just make a list of all the centerline endpoints and calculate the distance to the clicked markups point, and find the one with the minimum distance. For any markups node, you can get control point positions like this.
A function to take two centerlines and link them at the ends you specify. You can access the coordinates of each centerline point as in the previous section; so the only real complication is getting them in the right order. All of the points from one of the centerlines come first, and then all of the points from the other centerline, either in the original order or reversed, depending on which end is supposed to be connected.
A function to take two endpoints which are to be newly connected and which creates a new segment connecting them. If you want to base the radius of this segment on reported radii from VMTK, you also need a way of extracting that data (maybe look here, that isnāt something I have done myself before). With a radius and endpoints, you can draw a cylindrical tube using the āDraw tubeā segment editor effect. The new segment can then be merged with the two existing segments which are to be joined using logical operators āAddā segmentation effect. Some helpful code examples for logical operations can be found in this forum thread.
Then you would need to do whatever bookkeeping is needed (deleting the old versions of the merged segments and centerlines and updating with the new merged version)
Very helpful code snippets for all kinds of things in Slicer can be found in the script repository, this is one of the first places you should look when trying to figure out how to do something in Slicer using python.
There is also a LOT of information available in the Slicer Developer Guide documentation. It may be helpful to peruse areas of that site, though thereās far too much there to read and digest all in one go.
Thank you so much for such a detailed reply! This is really really helpful, and I will go through these in details. I already played around with the āDraw Tubeā effect today, but one thing I noticed is that inside the effect I always have to define new control points to draw the tube instead of just connecting two endpoints that are pre-defined. I now think that it might be easiest for me to firstly extract the endpoints of each segments and just let the user select which two endpoints to join by drawing a tube. Do you have any suggestion on how I may use existing fudicial/control points as input to the āDraw Tubeā effect? Or does this mean I will need to implement it in my custom module?
Here are the pieces I see that you will need: