# Composite transform from ANTsR is read incorrectly into Slicer

**URL:** <https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498>\
**Category:** Support\
**Created:** [November 29, 2022, 7:25pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498 "2022-11-29T19:25:21Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [November 29, 2022, 7:25pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/1 "2022-11-29T19:25:21Z")

</div>

I have registered a mouse image to a template in ANTsR (R version of ANTs). We have saved the resultant affine and deformable transforms and when these are loaded into Slicer and correctly nested, the resultant image is correct (i.e. consistent with what I obtain in ANTsR).

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/5/0/50057fb64f07eb6622cdc05beec07d3ac3d2eac3.jpeg)

From the affine and warp field, I can also a create a single composite transform in ANTsR. I know this transform is correctly written to disk, because I if I directly apply (read from disk) it to the image I get the same image in screenshot. Thus the issue is not on ANTsR end.

However, if I load the composite transform and apply it to the image, this is what I get in Slicer:

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/4/6/46120b9cfdf60ab58725d740e4c5bd237d7ba44a.jpeg)

Any ideas why this might be the case?

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [November 29, 2022, 7:42pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/2 "2022-11-29T19:42:10Z")

</div>

> [@muratmaga](#):
>
> Any ideas why this might be the case?

If it is helpful, there is this warning generated by Slicer specific to the composite transform:

> Warning: In /work/Stable/Slicer-0-build/vtkAddon/vtkOrientedGridTransform.cxx, line 335  
> vtkOrientedGridTransform (0xbad5f80): InverseTransformPoint: no convergence (0, 34.7143, 224.571) error = 0.491319 after 500 iterations. Further convergence warnings suppressed until transform is modified.

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [November 30, 2022, 4:24pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/3 "2022-11-30T16:24:35Z")

</div>

@simonoxen do you have any experience with getting composite transforms from ANTs/ANTsX loaded into Slicer? Or have you encountered an issue like this?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [December 1, 2022, 5:49am UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/4 "2022-12-01T05:49:49Z")

</div>

If you load the composite transform and click “Split” in the transforms module then do you see the same transform components as the ones you loaded separately? (maybe it is easier to compare them if you save to file)  
What ITK version is used in your ANTsR?  
Does thing work well if you use SlicerANTs?

> [@muratmaga](#):
>
> If it is helpful, there is this warning generated by Slicer specific to the composite transform:
> 
> > Warning: In /work/Stable/Slicer-0-build/vtkAddon/vtkOrientedGridTransform.cxx, line 335  
> > vtkOrientedGridTransform (0xbad5f80): InverseTransformPoint: no convergence (0, 34.7143, 224.571) error = 0.491319 after 500 iterations. Further convergence warnings suppressed until transform is modified.

If you don’t see major issues when you visualize the transform (e.g., using Grid mode) then probably you can ignore this message.

Can you upload somewhere the separate transforms, the composite transform, the fixed and moving image and the image resampled with ANTs?

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [December 1, 2022, 7:43am UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/5 "2022-12-01T07:43:55Z")

</div>

> [@lassoan](#):
>
> If you load the composite transform and click “Split” in the transforms module then do you see the same transform components as the ones you loaded separately? (maybe it is easier to compare them if you save to file)

I don’t see this button.

> [@lassoan](#):
>
> What ITK version is used in your ANTsR?

```auto
> itkVersion()
[1] "5.3"
> itkTag()
[1] "v5.3rc04"

```

> [@lassoan](#):
>
> If you don’t see major issues when you visualize the transform (e.g., using Grid mode) then probably you can ignore this message.

It is completely wrong. it somehow made into four images as shown in the screenshot.

> [@lassoan](#):
>
> Can you upload somewhere the separate transforms, the composite transform, the fixed and moving image and the image resampled with ANTs?

[here it is as an mrb file](https://app.box.com/shared/static/re8czcysnryxoqkufwdlfxzz1x4yq4cw.mrb)

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [December 1, 2022, 7:52am UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/6 "2022-12-01T07:52:16Z")

</div>

I forgot to include the ANTsR resampled image. here is the link for that. [https://app.box.com/shared/static/yk7vvhlaagcmkwlbl37s8y0hzjiqthss.nrrd](https://app.box.com/shared/static/yk7vvhlaagcmkwlbl37s8y0hzjiqthss.nrrd)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [December 1, 2022, 1:39pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/7 "2022-12-01T13:39:16Z")

</div>

Thanks for the data sets. I don’t see any issue on my computer (Windows10, Slicer-5.2.1). The Composite transform and the Warp+Affine transform result in the exact same transformation (which is the same as the Ants-resampled image):

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/6/8/68daa28fa7f0bfbbe4642fca7a363b0c06d9ffc3.jpeg)

Check it on a few different computers to see if you see anything common between those that don’t work correctly.

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [December 1, 2022, 4:02pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/8 "2022-12-01T16:02:29Z")

</div>

Interesting that’s not what I am seeing on my end on windows 5.2.1

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/b/a/baa941a195adf32333bc2e4a77e68bfd59f73502.jpeg)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [December 1, 2022, 8:18pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/9 "2022-12-01T20:18:51Z")

</div>

Try unzipping the scene files into a folder and load the .mrml file. Many things can go wrong when you open a 6GB zip file. Or it might be some rendering issue? Have you tried this on different computers/operating systems?

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [December 1, 2022, 11:53pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/10 "2022-12-01T23:53:42Z")

</div>

This happens to me both on Ubuntu 20.04 and windows 11 with 5.2.1

Can you make sure you zoom in, because if it simply centered on the reference image, you may not notice the deformations in the image. Better yet simply have only the moving image displayed, zoom in and slide across the stack. This is what I see:

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/b/6/b6a631157fd327d3e26d6b38b7d76b66d894351a.jpeg)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [December 2, 2022, 1:29am UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/11 "2022-12-02T01:29:17Z")

</div>

I see what you mean now. If you zoom out then then you can some artifacts far, far away from the transform’s domain. The artifacts are due to that it is a grid transform with very large displacement vectors at the edge of the domain. Such artifacts are completely normal and you can safely ignore them.

[![](https://img.youtube.com/vi/0YI9U5LCi4w/maxresdefault.jpg "Murat combined transform test 2") ](https://www.youtube.com/watch?v=0YI9U5LCi4w)

Why are there artifacts?

The displacement is extrapolated outside the transform’s domain in a way that:

1. The displacement vectors change changing at the edges of the transform domain (otherwise the transform would not be invertible, which is essential if you want to transform both images and meshes), and
2. At infinite distance from the transform’s domain the displacement converges to zero.

If a transform has very large displacement at the edge of its domain then the extrapolation may have some large waves before the displacements converge to zero. When you zoom out then you see those large displacement waves far away from the transform’s domain.

How to avoid such displacement extrapolation artifacts?

If you want to avoid such extrapolation artifacts outside the image domain then the best is to keep the large linear rigid bulk transform and the remaining warping grid transform separate. Then the displacement vectors in the grid transform will be small and if there is any extrapolation, the displacements converge to zero very quickly, without any waves.

ITK can save these two transforms into a single transform file (that is a composite transform), but maybe this option is not made available in ANTs. If ANTs cannot save composite transforms then you need to educate users about ignoring extrapolation artifacts or teach them how to load and apply multiple transforms.

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [December 2, 2022, 2:42am UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/12 "2022-12-02T02:42:37Z")

</div>

> [@lassoan](#):
>
> If you want to avoid such extrapolation artifacts outside the image domain then the best is to keep the large linear rigid bulk transform and the remaining warping grid transform separate.

Yes, that part is fine. When I apply the separate affine + warp transform as nested, I do not see those artifacts at all.

> [@lassoan](#):
>
> ITK can save these two transforms into a single transform file (that is a composite transform), but maybe this option is not made available in ANTs

It is available, and that’s what the composite\_transform is. That’s the one showing the artifacts when loaded into the Slicer.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [December 2, 2022, 2:54am UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/13 "2022-12-02T02:54:10Z")

</div>

A composite transform file contains multiple transforms. The transform that is named as “moving-Forward-compositetx” is a simple grid transform that contains the total displacement for each point of the fixed image. You can see this in the “Information” section in Transforms module:

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/c/e/ce816e72c1fabe5fac9fc874808d77383be2c733.png)

You can create a composite transform by hardening the warping transform on the bulk affine transform. The information section will look like this and the “Split” button will appear that allows the user to split the transform to its components:

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/c/c/cce3811f0871b3a31e14c99f2a89de76df0afb1c.png)

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [December 2, 2022, 5:13pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/14 "2022-12-02T17:13:09Z")

</div>

> [@lassoan](#):
>
> You can create a composite transform by hardening the warping transform on the bulk affine transform. The information section will look like this and the “Split” button will appear that allows the user to split the transform to its components:

This is the command in ANTsR (and ANTs proper as well) to concatenate transforms into a single transforms. that’s what I used to generate the composite transform.

`antsApplyTransforms(ref, moving, transformlist = c("moving-Warp.nii.gz", "moving-Affine.mat"), compose='./moving-Forward-compositetx')`

Your method of hardening separate transforms in Slicer should be equivalent to this at the end. But the fact that I am seeing deformation artifacts in composite from ANTsR but not from the composite transform by hardening in Slicer tells me that something is going wrong when the original composite transform is applied to the image. I am not going to chase this any further, but I wanted to bring this up in case someone encounters this.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [December 2, 2022, 6:27pm UTC](https://discourse.slicer.org/t/composite-transform-from-antsr-is-read-incorrectly-into-slicer/26498/15 "2022-12-02T18:27:52Z")

</div>

> [@muratmaga](#):
>
> This is the command in ANTsR (and ANTs proper as well) to concatenate transforms into a single transforms. that’s what I used to generate the composite transform.

Composite transform refers to a very specific thing in ITK: a transform that contains multiple transforms inside. Let’s use it only for this meaning to avoid misunderstandings. The `antsApplyTransforms` command does not create a composite transform.

The `antsApplyTransform` simply crates a grid transform by modifying a transform by another one. It is an irreversible, lossy operation, which fundamentally changes the properties of the transform (it may not be feasible to robustly invert or extrapolate it anymore), and may hugely increase the storage size (if the input contained warping transform, such as bspline or tps). There are some meaningful uses of this transform (it allows resampling an image in the transform’s domain without requiring someone to implement ITK transforms; or get a displacement field for statistical analysis), but in general, the `antsApplyTransform` command should be avoided for grid transforms.

> [@muratmaga](#):
>
> But the fact that I am seeing deformation artifacts in composite from ANTsR but not from the composite transform by hardening in Slicer tells me that something is going wrong when the original composite transform is applied to the image.

It seems that in your workflow ANTs is not used properly. “General registration (ANTs)” in Slicer (provided by SlicerANTs extension) provides a composite transform as a result. You can have a look at [SlicerANTs source code](https://github.com/netstim/SlicerANTs) to see how to achieve this.