Volume Reslice for Calibrated Ultrasound Images

Development
, , ,
1

Operating system: Windows 7 and MacOS 10.14.6
Slicer version: 4.10.1
Expected behavior: 1. Set the incoming tracked image as the driver for the corresponding 2D viewer.
2. Find the appropriate mode from the Mode list that keeps your full tracked image in the 2D viewer. (All mode settings tried).
3 (expected behavior). Visualize the whole US image in the 2D viewer in the 3D viewer by activating the “eye” icon in the header of the 2D viewer.

Actual behavior: 3. Only thin line visible in 2D viewer (and therefore in 3D view).

Additional Info: I can view the ultrasound image using the volume reslice driver when just applying the probe transform without a calibration transform (which is what the IGT instructions show). However, this shows the ultrasound in image space (with units = pixels), while all other scene information is in physical space (units =mm). The ultrasound calibration has a relatively large scaling aspect (e.g. 414 pixels -> 40 mm). The scaling can be accurately applied, but the image volume cannot be viewed after the calibration rotation is applied. Models, segmentations and points designated in the US images are properly transformed by the applied transforms, but the image always disappears when transformed. This leads me to believe that the calibration is not faulty, and this is a volume visualization problem.

Unsuccessful fixes (a short list): I believe it should work with the Transverse mode applied and the US image volume as the driver, but I also tried all combinations of reslice modes and drivers. I tried separating scaling and rotation matrices, with various combinations of hardened transforms so that the volume reslice driver was only accounting for one transform at a time.

Data if you want to see what I mean: ultrasound image volume and calibration transform (as h5, txt and mat file)

Is there a way to visualize this transformed image in the 3D view?

2

Which SlicerIGT tutorial have you followed?

Can you upload your Slicer scene and Plus configuration file somewhere and post the link here?

3

Thank you so much for replying so quickly!

The SlicerIGT tutorial I referenced above is here: Tracked ultrasound tutorial · SlicerIGT/SlicerIGT Wiki · GitHub

I’ve uploaded a scene and the data (a sample US image with the applied calibration transform) here.

Plus is configured to give a framegrabbed ultrasound (US) image and tracking info from NDI, and it does this successfully.

I’ve added further explanation and some pictures to help me better explain what’s going on. I’m trying to get a frame grabbed ultrasound image to display in world coordinates. The trouble happens when applying the probe calibration transform to an image volume.

I used an in-house calibration module to define the transformation from US image coordinates to the tracked US transducer (NDI tracking) in world coordinates. This gives a standard 4x4 calibration matrix. This matrix has low validation error – i.e. it generates points/models/segmentations in correct world positions. The calibration matrix can correctly transform other node types (fiducials, models, segmentations, etc.). Here I have fiducials (pink), a model (blue), and a segmentation(green) shown in the green slice view (in image space). Clones of these objects are made and transformed (with NDI transforms and the calibration matrix) in 3D space. A model of the ultrasound image bounds is also shown (rectangle). What’s important is that these are all in the correct space in the world coordinates.
2020-05-14-Scene (1)

Via the aforementioned tutorial, I can get a live image volume displaying, moving with the tracked probe no problem BUT without the calibration applied. Here you can see the image, transformed by an NDI tracking transform, and visualized in the scene using the volume reslice driver. I turned on the yellow slice view to show an orthogonal (not resliced) plane.
2020-05-14-Scene1 (1)

The image “disappears” when I apply a transform node containing our calibration matrix, except for one line of pixels. Here I have the volume reslice driver set to show Transverse, Inplane, and Inplane-90, and you can see each corresponding line of pixels.
2020-05-14-Scene1

Basically, I want to visualize my calibrated tracked ultrasound image in the 3D view, but I am not able to. For some reason, it seems I can transform and visualize any other kind of node aside from an image.

4

I wanted to bring this thread back up. Has a solution been found?

We have the exact same issue when applying a custom transform from our own calibration scripts. Tracking the US without a calibration transform applied works just fine. However, as soon as we apply our own transform the 2D US image is reduced into a line just as Winona

Please, let me know if there is a workaround.

5

If you don’t see the image in the slice view then it means that you have chosen an incorrect transform for driving the slice view in Volume Reslice Driver module. Probably the easiest if you keep the IJKToRAS transform that is embedded in the image as identity matrix, and you position the image using ImageToReference (=ProbeToReference*ImageToProbe) transform.