3D-2D CT-Fluoro registration

Support
1

Hello community,

I would like to perform 3D-2D registration (CT-fluoroscopy) on pelvis data from the GitHub repository GitHub - rg2/Regi2D3D-IPCAI2020: Code for the registration component of the IPCAI 2020 paper: "Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration." https://arxiv.org/abs/1911.07042 or https://doi.org/10.1007/s11548-020-02162-7.

I have some assumptions on how to perform the registration:

  1. Create DRR - I can achieve that with GitHub - arcadelab/deepdrr: Code for "DeepDRR: A Catalyst for Machine Learning in Fluoroscopy-guided Procedures". https://arxiv.org/abs/1803.08606.
  2. Compare two images to obtain the NCC loss (DRR, fluoroscopy).
  3. Optimize the transformation and rotation matrix to minimize loss. At the moment, I can use some nature-inspired algorithm since this is a proof of concept and real-time capability is not necessary yet.

I have read @lassoan’s answers from both Get projection plane in 2D-3D Registration and Get projection plane in 2D-3D Registration.

2D/3D registration is a challenging problem. You need to set the initial pose close to the solution, but there may be other tricks as well.

If I want to get the initial pose close to the solution, can I perform manual registration via some fiducials in 3DSlicer? If yes, could you please tell me how?

Thanks