Displacement field and deformable registration: Unexpected folded “mirror” images outside the fixed volume

Support
1

Operating system: Windows 7
Slicer version: 4.10.0
Expected behavior: Displacement field deforms moving volume only within the limits of the displacement field (i.e. within the limits of the fixed image). Regions outside of the displacement field should not affect the registration.
Actual behavior:

I have a problem with displacement fields (deformable registration) in Slicer: Using SlicerElastix, I register a (pig) cardiac CT [moving volume] to a (comparably small) ex vivo MRI of the explanted heart [fixed volume]. Due to post mortem tissue deformation, the deformation is rather large but Elastix handles this extremely well. The output from SlicerElastix is a displacement field, which I then apply to the moving volume. Registration results within the FOV of the fixed volume are nice. This workflow has worked without any problem in two cases in the past, but this time I noticed strange results:

When I look beyond the extent of the fixed volume, there is strange folding and mirroring happening. (See screenshot. Moving volume in blue, fixed volume in yellow, displacement field visualized in the 3d view)

screenshot1
screenshot11920×1170 962 KB

This would not matter if this folding would not mess up the transformation of segmentations. I do have surface models (isodose volumes from SlicerRT) as well as a segmentation, both mapping to the moving volume CT and both located in a region which should actually be registered into the fixed volume. However, both structures become grossly deformed and do not display correctly due to the folding happening outside of the fixed volume.

To illustrate this effect, I have created an affine (non-deformable) registration which approximates the deformable one. This is how the isodose lines, the segmentation, and the moving volume (blue) look under this affine transform:

screenshot2
screenshot21920×1170 703 KB

However, once I apply the displacement field instead of the affine transformation, the isodose volumes / isodose lines and the segmentation become wildly distorted and do not display at all in some views. Note how the green slice goes through the white segmentation as can be seen from the slice intersection in the other slice viewers. Nonetheless, no white segmentation is displayed in the green slice viewer. Note also how the white segmentation (in the yellow and red viewers) is duplicated / mirrored in regions outside the actual fixed volume, leading to an wildly stretched appearance in the 3d view.

screenshot3
screenshot31920×1170 826 KB

I wondered if it has something to do with the size difference between the fixed and the moving volume and with the displacement field being undefined in certain places. I have already tried to either crop the moving volume to the size of the fixed volume or to overcrop the fixed volume so that it is the same size as the moving one. The result have been the same (folding, mirroring, chaos).

What am I missing?
Any hints are greatly appreciated.

Thank you very much

Stephan

2

It is very important to estimate displacement field outside the limits of the fixed image, because transforming points and surfaces require inversion of the resampling transform (that is use for transforming images). If you don’t have a smooth boundary but displacement field would abruptly change from some large value to small value then the inverse computation would fail.

If you have very large displacements (especially rotations) then you may perform rigid registration first and harden the results (this does not cause any change to the image voxels, just changes position and orientation of the voxel grid). After this initial alignment, you should have no problem with performing deformable registration.

Note that it is very rare to have flips or large rotations between registered volumes and it may indicate problems in previous steps of your workflow. So, if you regularly encounter such cases then probably it is better to fix your imaging or pre-processing workflow.

3

That was it. I realized that in the two previous cases (which worked well) I had hardened any linear transform on the moving volume before starting Elastix, while this time I simply used those linear transforms to initialize Elastix. This way, the large translation and rotation* ended up being a part of the displacement field this time.
I did not know that displacement fields are so sensitive to large displacements, so thank you for pointing this out.
And thank you for the very quick reply, as always.

Stephan

Because you mentioned this: The large amount of translation and rotation stem from the fact that the moving and fixed volume were acquired under very different circumstances, one being a CT of a living animal and the other being an MRI of the heart in a jar… So it is challenging to register them, indeed.

1 Like