I’m working on a combined visualization of an preoperative MRI image and intraoperative thermography images. The intraoperative thermography image (2D) is registered to the MRI image with a tracking system that utilises OpenIGTLink (the reason for this thread). Afterwards I create a visualisation of the thermography images by re-projecting them onto a surface that is reconstructed from the MRI image. This resulting 3D visualisation of the thermography image should be combined with an arbitrary visualisation of the MRI image. I plan to do this using image fusion.
I split up the whole visualisation pipeline into the following modules:
Streaming module that takes care of receiving the intraoperative thermography images and the tracking data (done)
Re-projection visualisation module that creates the 3D visualisation of the thermography images
Image fusion module that creates a single visualisation out of to the thermography visualisation and an arbitrary MRI visualisation (e.g. Direct Volume Rendering)
I’ll create different threads for the Re-projection and the Image fusion module
Image fusion module = you set MR image as background and thermography visualization image as foreground image; you can tune colormap and thresholding setings in Volumes module
I would strongly recommend to complete all SlicerIGT user tutorials at http://www.slicerigt.org/wp/user-tutorial/ to get an idea what Slicer/SlicerIGT can already do, but of course we are happy to answer any specific questions here.
Thanks for your suggestions and creating a thread for me!
I looked at the tutorials and they don’t describe what I plan to do.
As I understand Volume Reslice Driver module positions a 2D image in the 3D scene given a continously updated transformation. That is not what I am trying to do.
The Re/Back-projection visualisation module should project (maybe re-projection is a translation error) the 2D image back onto a given 3D surface. Imagine a beamer that projects the 2D thermography image onto the surface that it depicts. As a result, each surface point should be colored according to its temperature. Slicer is able to create such a surface from the MRI image. E.g. for neuro surgery there are skull stripping extensions and surface models can be created from label maps / volumes.
vtkProjectedTexture achieves the same effect, but on vertices (not fragments). That is why, after implementing a prototype using only VTK, I decided to create a custom mapper. This mapper takes a model, an image and camera parameters and creates this desired visualisation (easy and fast GPU based implementation). As far as I understand Slicer and VTK I propapbly need to create my own display node type and a matching displayable manager.
For the image fusion module I was a bit short on description as well. I want to fuse two different 2D images / image sources that are based both on a visualisiation similar to the 3D view. I want to perform an off screen rendering of both 3D visualisations into 2D images. These images should then be fused using an algorithm similar to the one described in this paper http://ieeexplore.ieee.org/document/6423909. The result should then be displayed on a camera aligned plane in the 3D view.
Some more information about me: I have a computer science background and I am mostly new to the medical imaging domain. The whole project I described is part of a minor thesis, that I am currently working on. The focus is more one the theoratical background and less on practical day-to-day usability.
Thanks for the clarification. I assumed that you use MR thermometry and use slice visualization. Volume reslice driver module just reslices volumes, so it is not what you need.
Your project sounds very exciting. There will be a project on medical infrared imaging on the upcoming Slicer Project Week at MIT in January by @carlos-luque and others. It would be great if you could update us about your progress, or even come to the project week - you could accomplish a lot during a week there.