How to implement real-time surgical navigation by mapping Intel RealSense D405 RGB-D camera with pre-operative CT/CBCT in 3D Slicer?

Hello everyone,

Thank you for your kind attention.

We are developing a real-time intraoral surgical navigation system using 3D Slicer and would appreciate guidance on the recommended workflow.

Objective

Our goal is to overlay a live RGB-D camera view (Intel RealSense D405) onto a pre-operative CT/CBCT scan so that the live surgical scene is accurately mapped into the CT coordinate system.

The final system should provide:

  • Live RGB image
  • Live depth image
  • Real-time point cloud
  • CT/CBCT volume
  • Live camera pose relative to CT
  • Continuous visualization during surgery

Current Hardware

  • Intel RealSense D405

For example :slight_smile:

We don’t have a sensor at the end of the suction tip. We want to visualize the live 2D operation view together with the CT image in 3D Slicer. Should we purchase a specific camera for this purpose, or do we need an additional sensor? If we use a sensor, the suction tip position and operation should be highlighted in both the live 2D view and the corresponding CT image. Thank you.