I am working on the slice-to-volume registration topic using ITK. Conventionally, slice-to-volume registration (e.g. US-CT) needs to reslice the volume data based on the transformation and then convert it to a 2D-2D registration problem. In this discussion, TwoProjectionRegistration was suggested to kick off the slice-to-volume registration. However, there are still lots of differences.
I wonder whether there is an existing “interpolator”, which can achieve the volume data reslicing and interpolation tasks, similar to the “itkRayCastInterpolateImageFunction”. I think that if this interpolator exists, we can connect with other components (similarity metric, optimizer) to achieve the slice-to-volume registration. If not, any suggestions to write one?
Slice-to-volume registration does not require any special interpolator, as it directly aligns voxels of the slice to the voxels of the volume. Single-slice-to-volume registration is a simple 3D-3D registration task (just one of the 3D volumes happen to have only a single slice). The only thing you need to pay attention to is to use the single-slice image as fixed image (otherwise ITK will return with the error that most voxels of the fixed image lie outside of the moving image).
@lassoan I tried the solution you mentioned, and it worked well. I really appreciate it!
For now, I have two more questions about the slice-to-volume registration, I hope you could provide me with some ideas:
(1) In SimpleITK/ITK, there are some existing similarity metrics to facilitate the registration task. If I wish to develop my own similarity metric, instead of rewriting a new similarity metric API (based on C++), is there a more straightforward solution available? I am guessing that my own similarity metric has to integrate with the well-streamlined registration workflow in SimpleITK/ITK, It seems that writing a new API, similar to other existing similarity metrics, might be the only option.
(2) From my understanding, similarity metrics are mostly calculated based on the entire image pairs or masked image pairs, is it possible to calculate patch-level-based similarity metrics? In other words, can I randomly choose many patches (let’s say 1000) from the fixed and moving images respectively, and then evaluate the patch-level-based similarity metric for the following optimization? I am not sure how these patches integrate into the registration workflow.
Similarity is typically evaluated on a set of sample points, e.g., 10% of the fixed image voxels.
There are many metrics in ITK already, so you may not need to implement yet another one. But if you want then you can write a new one or combine existing similarity metrics in C++ for sure. Maybe in Python, too, but you can get definitive answer for this on the ITK forum.
@lassoan , thank you for your quick response and for clarifying my misunderstanding. I will certainly look into it.
Regarding question 2, which I previously inquired about, I am also wondering how I can calculate patch-level-based similarity metrics. Do we also need to write a new function to break images into small patches and integrate them into the registration workflow? Any thoughts about that?
There has not been much work on patch-based similarity metrics in ITK, most likely because intensity-based metrics work so well for medical images (see for example this old discussion). SIFT based registration was evaluated for challenging registration tasks (e.g., ultrasound slice registration to ultrasound or MRI volume) with some success, but these are niche applications, so I’m not sure that they have made it into any commonly used registration toolkits.
You can implement any similarity metric, including patch-based metric. Since the metric has to be evaluated at hundreds of thousands or millions of positions in each iteration and typically you need hundreds of iterations, you probably have to do it in C++ (perhaps with some CUDA acceleration) for a reasonable computation time.
There are many registration experts in the ITK forum, so I would recommend posting further questions on this topic there.