# CT to optical image registration

**URL:** <https://discourse.slicer.org/t/ct-to-optical-image-registration/514>\
**Category:** Support\
**Created:** [June 16, 2017, 2:28am UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514 "2017-06-16T02:28:20Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [June 16, 2017, 2:28am UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/1 "2017-06-16T02:28:20Z")

</div>

Hi there,

I have 2 image sets: one is a high resolution CT from a small specimen with  
0.15, 0.15, 0.15 millimeter spacing in each of the X, Y and Z axis, the  
other one is an optical stack of the same specimen taken using a digital  
camera with the following spacing (mm) in x,y,and z axis respectively:  
0.05, 0.05, 3. The photographed images are taken carefully so there is a  
corresponding CT slice for each optical image.

I would like to align the two image sets.

I’m using Slicer 4.6.2 and tried landmark registration but when I save the  
transformed image (hardened), and then open it, it doesn’t match with my  
fixed image!

Can you help me do the registration properly?

Thanks,  
Niousha

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [June 16, 2017, 3:30am UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/2 "2017-06-16T03:30:12Z")

</div>

Could you please try it with the latest nightly build?  
Could you attach a few screenshots of how the images look before registration, after registration, after hardening, and after reloading the hardened image?

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [June 17, 2017, 1:46am UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/3 "2017-06-17T01:46:11Z")

</div>

Hi,

Please see attached for the images requested.  
Fixed image: optical (0.05, 0.05, 3 millimeter spacing)  
Moving image: Ex vivo ( 0.15, 0.15, 0.15 millimeter spacing )

P.S. The CT images were reoriented using Inveon software prior to  
registration( before reorientation image not included).

Thanks,  
Niousha

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/f/5/f51fd5143aad09036db15bbf3cd31a2f4c562f38.png)

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/1X/e5be58016c034e4a83a91f6839c3546b163aad9d.jpg)

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/6/2/62fd9f76104ece085d6f12bef515e7a1d4b910e5.png)

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/3/4/340341278e681deed777a7935861e7dd7b42134c.png)

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/4/4/448b2c1bd0e03895495656cdba5cc3358c44a9cf.png)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [June 18, 2017, 6:43pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/4 "2017-06-18T18:43:18Z")

</div>

Could you attach a few screenshots of how the images look **in 3D Slicer!!!** before registration, after registration, after hardening, and after reloading the hardened image? Many software make mistakes when importing images (ignore image orientation, etc), so we have to make sure the registration is correct within 3D Slicer, and if it is, then you can report the errors to developers of other software packages that you use. By the way, as you might have realized, Slicer can provide registration and visualization features that fulfills most users needs; so probably you can simplify your life by doing all processing in Slicer.

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [June 19, 2017, 9:23pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/5 "2017-06-19T21:23:00Z")

</div>

Hi Andras,

I tried the latest version and also tried _not to center_ the images this  
time, and it worked (previously the images were centered before  
registration). So I got good alignments when I set the fixed and moving  
image as follows:  
Fixed image: optical (0.05, 0.05, 3 millimeter spacing)  
Moving image: Ex vivo ( 0.15, 0.15, 0.15 millimeter spacing )

But the problem for now is to do the reverse registration. In other words,  
I’m interested in doing the registration in such a way that my optical  
images are moving and my CT ex vivo images are fixed (the spacing is the  
same as indicated above). When doing that it shows me good overlap but I  
can’t save the hardened image. Please see the Error message attached; I  
also attached a few screen shots of before and after the alignment. I also  
included a screen shot of after hardening. But I don’t have an image after  
reloading the hardened image, because, as I explained, the hardened image  
couldn’t be saved. But I included the message generated after trying to  
save it. Based on your knowledge, how would I be able to perform such  
registration in Slicer?

I also noticed in our last email conversation you suggested that Slicer can  
work for most of the analysis and visualization. So my question is can I  
calculate the registration errors based on measures of Mutual Information  
and Target Registration Error using this software? If yes, can you please  
provide some help on how this is done?

Thank you,  
Niousha

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/5/3/53c1ea1a9c1a6dd90941cf40f5899e016e601d6a.png)

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/7/7/77e435570dd14c649f05351a8f65c75bd1af9512.png)

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/1/c/1c0f03f836733be977cff96f13a626579582ec29.png)

 ![](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/6/2/62794f26427e6928af8a9bbb67653a1051ee9026.png)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [June 20, 2017, 12:48am UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/6 "2017-06-20T00:48:07Z")

</div>

> [@Niousha\_Aflatouni](#):
>
> tried not to center the images

Never center images if you need proper alignment. If centering is enabled then it forces Slicer to ignore image origin information that is stored in the image header.

> [@Niousha\_Aflatouni](#):
>
> to do the reverse registration

Slicer can compute the inverse of any transformation (linear, bspline, thin-plate spline, displacement field; or any combination of these in any order, with/without inversion in each component) by a single click. Go to Transforms module and click `Invert` button in `Edit` section. You can then align the fixed image to the moving image by applying the inverse transform to the fixed image.

In Slicer you can choose from a wide range of image registration methods: intensity-based, landmark-based, surface-based; with various metrics and transformations. For intensity-based registration with warping transform I would recommend to try these:

- [General registration (Elastix)](https://github.com/lassoan/SlicerElastix/blob/master/README.md): available in the SlicerElastix extension. It is very robust and simple to use (requires much less parameter tuning than BRAINS).
- [General registration (BRAINS)](https://www.slicer.org/wiki/Documentation/Nightly/Modules/BRAINSFit): available in the Slicer core, by default uses mutual information. There are lots of examples and parameter sets in the [registration case library](https://www.slicer.org/wiki/Documentation/Nightly/Registration/RegistrationLibrary).

Compute TRE: Mark landmarks in Fixed and Moving volume using `Markups` module. Apply computed transform to landmarks of Moving volume. Write a short script (2-3 lines of Python code) that compute distance between Fixed landmarks and transformed Moving landmarks.

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [June 21, 2017, 8:26pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/7 "2017-06-21T20:26:23Z")

</div>

Hi Andras,

I tried the inverse transform the way you explained and I can see the  
change in the image but I still have the same issue as before: the hardened  
image can not be saved and the same error message appears. I couldn’t  
figure out what the issue is. Any thoughts on that?

As for the intensity based registration, I’m more interested to know if  
there is any way that the registration can be quantified (not performed)  
based on Mutual Information, which is an intensity measure?

Thank you,  
Niousha

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [June 21, 2017, 10:02pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/8 "2017-06-21T22:02:02Z")

</div>

> [@Niousha\_Aflatouni](#):
>
> the hardened  
> image can not be saved

Saving should be no problem, but maybe if it’s a color image there may be issues. Could you try if saving works if you convert the image to scalar (grayscale) volume (using Vector to scalar volume module)?

> [@Niousha\_Aflatouni](#):
>
> As for the intensity based registration, I’m more interested to know if  
> there is any way that the registration can be quantified (not performed)  
> based on Mutual Information

Mutual information minimizes the entropy of the joint histogram. You can find the final value in the application log (the metric value should decrease in each iteration), but it’s not a metric of registration quality.

Registration is typically assessed using spatial error metrics. If you have ground truth landmark points then compute TRE. You may also compute Hausdorff distance using Segment Comparison module (between ground truth segmentation in the fixed image and segmentation in moving image transformed to fixed image coordinate system).

The only intensity-based metric that I’m aware of that might be somewhat applicable for assessing registration (or, more accurately, the effect of remaining registration error) is the [module for comparison of dose volumes (in SlicerRT extension)](https://www.slicer.org/wiki/Documentation/Nightly/Modules/DoseComparison).

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [June 29, 2017, 8:45pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/9 "2017-06-29T20:45:45Z")

</div>

Hi Andras,

Thanks for the info. After many trials, I realized that the images should  
be cropped and turned into gray scale both in 3D Slicer, and if those are  
initially done in MATLAB then they won’t work in 3D Slicer.

I also found a feature in 3D Slicer called Metric Test, that seems to  
enable us to calculate MMI and MSE. There was however no documentation for  
this feature that I could find. I was wondering is this something that I  
can reliably use for calculating the MI? I used two identical images with  
exact overlap. For that I used different histogram bin numbers that ranged  
from 50 to 1000. Then the MI values I got ranged from -0.925 to -1.79812. I  
was wondering how can one interpret these numbers, also what is the  
suggested histogram bin number to use?

Thanks,  
Niousha

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [June 30, 2017, 1:42pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/10 "2017-06-30T13:42:55Z")

</div>

> [@Niousha\_Aflatouni](#):
>
> if those are  
> initially done in MATLAB then they won’t work in 3D Slicer

What do you mean? What do you expect to happen and what happens?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [June 30, 2017, 1:53pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/11 "2017-06-30T13:53:56Z")

</div>

> [@Niousha\_Aflatouni](#):
>
> There was however no documentation for  
> this feature that I could find

I think the only help is in the tooltips. Let us know if anything is unclear.

> [@Niousha\_Aflatouni](#):
>
> I was wondering is this something that I  
> can reliably use for calculating the MI

Yes.

> [@Niousha\_Aflatouni](#):
>
> I was wondering how can one interpret these numbers

It is the metric that is minimized during registration. Read about the metric in journal papers, such as [https://www.researchgate.net/publication/222899257\_BRAINSFIT\_Mutual\_information\_registrations\_of\_whole-brain\_3D\_Images\_using\_the\_insight\_toolkit](https://www.researchgate.net/publication/222899257_BRAINSFIT_Mutual_information_registrations_of_whole-brain_3D_Images_using_the_insight_toolkit).

> [@Niousha\_Aflatouni](#):
>
> what is the  
> suggested histogram bin number to use?

Usually registration is not very sensitive to the number of bins. You can probably use the default value or slightly change it and see if it makes results better. See more details of mutual information metric in journal papers.

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [July 10, 2017, 7:30pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/12 "2017-07-10T19:30:13Z")

</div>

Hi Andras,

Thanks for your previous answers. I had some questions about the Metric  
Test to calculate MI; I’m interested in obtaining the MI value of a  
specific region from the images rather than the entire image sets. I  
understand that this normally can be done using Masking but I couldn’t find  
a place where we can input the masked images in addition to the fixed and  
moving image. Is this something that can be done when running the Metric  
Test for MI? If not, what are the alternatives?

Thanks,  
Niousha

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 11, 2017, 2:39pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/13 "2017-07-11T14:39:59Z")

</div>

You have two options:

- Option A: Extend the Metric Test module to accept mask images. You need to build Slicer from source code and be somewhat familiar with C++ programming (so that you can find out where to add the masking option).
- Option B: Create a simple scripted module that computes the metric at different positions. You can check out SimpleITK examples for registration [here](https://github.com/InsightSoftwareConsortium/SimpleITK-Notebooks/tree/master/Python) (for example, a [registration initialization example](https://github.com/InsightSoftwareConsortium/SimpleITK-Notebooks/blob/master/Python/63_Registration_Initialization.ipynb))

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [July 11, 2017, 8:03pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/14 "2017-07-11T20:03:35Z")

</div>

Hi Andras,

Can you provide some instructions on how to access the source code? I  
followed the instructions provided in here:  
[https://www.slicer.org/wiki/Documentation/Nightly/Developers/Build\_Instructions#CHECKOUT\_slicer\_source\_files](https://www.slicer.org/wiki/Documentation/Nightly/Developers/Build_Instructions#CHECKOUT_slicer_source_files)  
but it wasn’t clear to me what should be done next and how to access the  
(i.e. Metric Test’s )source code from there?

I’m really new to this so your helps are really appreciated.

Thanks,  
Niousha

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 11, 2017, 8:21pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/15 "2017-07-11T20:21:28Z")

</div>

After build is complete, you’ll see the source files (in c:\D\S4D\BRAINSTools\BRAINSFit\PerformMetricTest.cxx).

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [July 26, 2017, 3:24pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/16 "2017-07-26T15:24:52Z")

</div>

Hi Andras,

Do we need to do any pre-processing step before we import images that are  
modified in MATLAB?

The problem is anytime I do some modification on my .jpg images in MATLAB  
(i.e rotation, gray scale, cropping), then those images will change in  
strange and unexpected ways when exported back to Slicer.

Thanks,  
Niousha

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 26, 2017, 5:11pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/17 "2017-07-26T17:11:57Z")

</div>

Do you work with color images? 2D or 3D?

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [July 26, 2017, 6:24pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/18 "2017-07-26T18:24:31Z")

</div>

Hi Andras,

Yes, the images are colored, 2D initially. So what I do is I align them in  
MATLAB using control point registration, then save each individual modified  
2D image and then stack those in 3D Slicer. I then change the stack into  
gray scale in 3D Slicer.

Niousha

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 28, 2017, 9:35pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/19 "2017-07-28T21:35:01Z")

</div>

> I align them in MATLAB using control point registration, then save each individual modified  
> 2D image and then stack those in 3D Slicer

This should work. What problem do you see? If you just browse the aligned images in an image viewer, do they appear correctly, nicely aligned?

---

<div class="post-metadata">

**Author:** ![Niousha\_Aflatouni](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@Niousha\_Aflatouni](https://discourse.slicer.org/u/Niousha_Aflatouni)\
**Post date:** [August 16, 2017, 4:46pm UTC](https://discourse.slicer.org/t/ct-to-optical-image-registration/514/20 "2017-08-16T16:46:51Z")

</div>

Hi Andras,

I kind of figured out what the problem was. I should have used imwrite to  
save the images properly to avoid this issue. But thanks for following up.  
I was wondering does slicer support combining multiple transforms that can  
be applied to an image? For example, consider having 4 images, (i.e. image  
1 to image 4), with the ultimate goal of mapping image 4 such that it  
aligns with image 1, can we find a final transformation matrix in 3D Slicer  
such that when we apply that to image 4 we can match image 4 to image 1?  
Please note that we already have obtained the transformation matrices  
between, image 2 and 1, image 3 and 2, and image 4 and 3.

Thanks,  
Niousha

[Next page](https://discourse.slicer.org/t/ct-to-optical-image-registration/514.md?page=2)