I would like to create a set-up in which I use a tracked-US system for intraoperative navigation using the open-source Aruco tracking software from OpenCV. Therefore, I connected my US system via an Epiphan framegrabber and OpenIGTLink to Slicer. Now I am working on building the tracking set-up. As I will use the system for intraoperative guidance, high accuracy is acquired.
Therefore, I created a file with two cameras as input (1. Epiphan 2. Webcam). However, the FCal software is not able to work with this set-up as created. Moreover, I am doubtful about the accuracy of the Aruco tracking. Especially, the in-plane movements such as translating towards/away the camera. Anyone who could give me advice about which camera (Intel realsense devices, LogiTech webcams) to use for minimizing error rates? Or, is there no alternative of buying an optical tracker such as the Optitrack Duo?
Could anyone help me with tips or advice what to do?
If you need high 3D tracking accuracy then you must use something better than a single-camera tracking using Aruco markers. Aruco is used good for setting up demos for people who don’t want investing any money into buying a tracker. Aruco is also fine for many augmented reality applications, because tracking error in the image plane is small. However, out-of-plane translation and rotation measurement is very inaccurate, so you won’t be able to get accurate 3D tracking out of it (you can have several centimeters error in 3D).
An Optitrack Duo has magnitudes smaller error and costs just $3000, so maybe it doubles the cost of your hardware setup, but does not make it a different magnitude. In theory, by optimizing Aruco markers, combining it with a RealSense or other 3D vision systems, etc. you might get acceptable accuracy, but this may take years of R&D work.
If the main goal of your project is not to develop a lower-cost position tracker then probably you are better off buying a tracker that is already accurate enough.
I forgot to mention our clinical case. We would like to image the tibia intraoperatively with tracked-US. We will move the probe mostly in the lateral direction by setting-up the camera perpendicular to the tibia.
In summary, the major limitation of the Aruco tracking is the out-of-plane translation and rotation. Could these limitations be overcome with the newest RealSense depth cameras with larger accuracy and the use of five Aruco markers in a 3D configuration as suggested by Ošˇcádal et al. (2020). They showed an increased out-of-plane translation accuracy by updating the middle QR-code with the tracking information of the surrounded ‘skewed’ ones. What is your opinion about using the newest RealSense cameras and this 3D Aruco marker configuration in relation to our clinical user case?
I hope this improvement could result in avoiding the need for an expensive optical tracking system :).
Our initial experiments showed that by using the surface provided Intel RealSense cameras we could achieve one magnitude smaller distance error than with a simple monoscopic camera, but it was still several millimeters, instead of submillimeter that you get from a proper tracker. More work would have been needed to fuse the surface data with the 2D image based tracking data. Making the markers larger and using more markers in a 3D configuration always helps, but there is a limit in the size and weight of markers that are tolerable in the OR. Visible-light cameras are also more impacted by strong lighting in the OR (as opposed to cameras that come with their own infrared light sources and matched filters on the camera). I would estimate that it would require a few years of research work and some luck to sort out all these issues and get similar accuracy from a RealSense camera as you get from a wider-baseline infrared stereo camera such as the Optitrack Duo.
Intel dropped the RealSense product line, so by the time you would have a solution, you would need to look for another camera. Camera systems similar to RealSense cost about $1000 (and they may disappear from the market anytime), so the cost saving is likely to become even smaller than today. It may make sense to invest time into improving Aruco tracking if your main goal is to reduce the navigation system cost by $2000 and development (and redevelopment, when your chosen camera is discontinued) time and cost does not matter much.
