I created a new extension for viewing registrations. It implements what is sometimes called “synchronised navigation” - simultaneous visualization of the same spatial location across multiple image series
The basic idea is that we load fixed and moving images, the corresponding registration, and when we move the cursor in one image, a markup follows in the second image (the position of the markup is calculated based on the registration).
Here is a sort video demo:
Some PACS providers, like Sectra, implement this feature (either via slices manually linked by the user or by a commercial registration algorithm).
I have already been using this with a few radiologists, and they seem to like it (when the registration is accurate).
Is this interesting to this community?
If yes:
Should it have more features?
Should it be a standalone extension, or does it make sense to integrate it somewhere?
This is in some way similar to this concept Views synchronization after registration, however, the main difference is that it shows the original moving image. As far as I understand, radiologists don’t always want to see a deformed moving image.
The code for anyone interested is available here:
How did I implement this (in short)?
create 6 vtkMRMLMarkupsFiducialNode (one for each view)
show the nodes in all views except for the one where the cursor is
use SetSliceOffset() to set the offset of all views to the position of the corresponding Fiducial node (except for the one where the cursor is)
for the views from the second image, first transform the corresponding fiducial nodes with the transformation and then set the offsets)
One small feature that I included is that when the user double clicks on a view, e.g., the Red one, a compare view opens with the Red and Red+ views side by side.
But did you look at the CompareVolumes module that already exists in Sllicer? As far as I know it already has the features you described, except the part where you show the correspondence but not the transformed volume. That sounds nice.
Plus CompareVolumes offers several other features, like an arbitrary number of volumes (selectable from all the loaded volumes, with the option to drag-and-drop to specify the order of the display), hot-linking pan/zoom/scroll operations, some animation modes for cross-fading between volumes, and a ‘compare cursor’ that provides an optionally magnified checkerboard inspection tool.
CompareVolumes was created to support some earlier research and it could certainly benefit from improvements, so maybe we could join forces?
I’d love to join forces on the Compare Volumes module!
I could fork your repo and implement the synchronised navigation feature, but I think we have to discuss a few things
Synchronised navigation feature
Initially, I thought I could implement it with the built-in crosshair, but for this feature, two decoupled crosshairs would be needed - one for the fixed and one for the moving. As far as I understood, you can only have one crosshair; creating another one would involve significant changes to the MRML Lib in Slicer (I think in vtkMRMLCrosshairDisplayableManager)
Therefore, I implemented it with markups for each view, which can then be transformed with the registration- it works well, but seems a bit hacky - do you have a better idea for how this could be done?
UI
How should the user choose the transformation?
How should the user choose the fixed and moving images?
How should the user enable the synchronisation (I would also like to have a keyboard shortcut in addition to a button)
What’s the nature of your transformations and what is the area of interest for your research (i.e. torso CT mainly?). That is, what is your target use case, since things like UI options and keyboard shortcuts are best considered in the context of a concrete clinical application.
I agree it’s not good to show clinicians distorted scans, as that could lead to misinterpretation. And yes, showing the corresponding points using two crosshairs would make sense, but using a markup as a fallback is also not a bad idea. If you look at the LandmarkRegistration module (code here), which uses the CompareVolumes logic under the hood, there is some code for managing per-volume markup lists of corresponding points. This way they can be moved interactively in either view and the other views are update through the transform to show the corresponding point. It’s probably very similar to the code you have, and maybe we can factor it out for reusability.
Another thing I’ve been playing with related to this is using segmentation results (e.g. from TotalSegmentator) to define correspondence between scans even when the patient’s body position is different. I.e. if you are looking at a tumor that’s a few centimeters from the femur, then registering the two femurs with a rigid transform puts the two tumors in close approximation even if the hip angles are very different in the two scans. There’s a prototype implementation here, where you can pick the anatomy you want to use for the local registration. Of course the registration is very good close to the rigid structures but can be very bad far from the selected structures. One feature I considered is that when you pick a point to compare, the either the single closest structures is used, or maybe the nearest few are used to define a blended rigid transform.
It would be great if we could develop layers of functionality, from general utilities through very dedicated task-specific interfaces.
Until now I used my extension for rigid and deformable transformations - I was hoping to make a general tool for any transformation type.
I guess you’re right, let’s not focus on the UI options and shortcuts for now, I can always just create them for my specific use case.
I’m currently evaluating registration algorithms with this tool in a reader study with a head & neck CT dataset (the example from my video was just from the Learn2Reg LungCT dataset as I can’t share my head & neck data).
The local registration seems really interesting, my radiology Prof already mentioned multiple times that he’s usually only interested in registration at specific sites of interest. Could you update the link to the prototype implementation of the local registration? I get a 404 error, maybe this repo is private?
Then my question is what should be our or my next steps?
I could take a look at the LandmarkRegistration code and see if there is any common code to factor out.
Ah, right - the repo was private since it’s a work in progress, but I made it public now.
In terms of next steps, it would be great if you could try the CompareVolumes on some of your data an registration results and see if the features and Lib code are a good fit for what you need. For example, in the LandmarkRegistration it sets up three rows of axi/sag/cor with the fixed, moving, and blended fixed/moving, which I find helpful for reviewing segmentation. You can do similar things with CompareVolumes by setting one of the volumes as the common background and unselecting it from the checklist. But I think the workflow is probably not so intuitive and maybe we can repackage into a dedicated one-to-one registration review module, like your current version but with more features.
