Cannot perform Aruco marker tracking

Support
1

I would like to try optical marker tracking which presented in this slides https://github.com/PerkLab/PerkLabBootcamp/blob/master/Doc/day2_Plus.pptx.

Using USB webcam, and Aruco marker, I would like to perform optical marker tracking demo like youtube video.

I basically follow the instructions written in the pptx. When I launched Plus server using configulation file attatched below (which was editted as same as the pptx said.), and connect the Plus server via OpenIGTLinkIF, nothing is shown in Scene. The slide #55 and #56 said that I should find those transform nodes "Scene > IGTLConnector > IN > Marker0ToTracker, Marker1ToTracker, Marker0ToMarker1 " .

I seemed to stack around slide #56.

Do you have any solution for this problem?

capture
capture1920Ă—1080 246 KB





















2

The transforms won’t appear in OpenIGTLinkIF until you show them to the camera.

Make sure to try a couple of different PortUsImageOrientation, MF/UF/UN/UF, depending on your camera, this may need to be adjusted.

Could you attach the Plus log and config file?

3

Thank you for your reply!

I’m using Windows 10 (64bit) PC with USB webcam.
Slicer 5.2.2
PlusApp-2.9.0.20230704-Win64

My config file is below

capture3
capture31893Ă—1035 205 KB

capture4
capture41900Ă—1016 155 KB

And the log file which was obtained immediately after launching Plus server using the config file above.

time|level|timeoffset|message|location
071423_192354.498|INFO|000.001000|> Software version: Plus-2.9.0.5a7cefc6 - Win64| in vtkPlusLogger(52)
071423_192354.500|INFO|000.002000| Logging at level 3 (INFO) to file: C:/Users/MasatoshiOba/PlusApp-2.9.0.20230704-Win64/data/071423_192354_PlusLog.txt| in E:\D\PTNP64b\PlusLib\src\PlusServer\Tools\PlusServer.cxx(81)
071423_192354.500|INFO|000.003000| Selected US image orientation: UN| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusDataSource.cxx(300)
071423_192354.501|INFO|000.004000| Server status: Reading configuration.| in E:\D\PTNP64b\PlusLib\src\PlusServer\Tools\PlusServer.cxx(92)
071423_192354.501|INFO|000.004000| Server status: Connecting to devices.| in E:\D\PTNP64b\PlusLib\src\PlusServer\Tools\PlusServer.cxx(103)
071423_192354.628|INFO|000.130000| Use aruco camera calibration file located at: C:/Users/MasatoshiOba/PlusApp-2.9.0.20230704-Win64/config/OpticalMarkerTracker/realsense_gen2_calibration.yml| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(233)
071423_192355.634|INFO|001.137000| Server status: Starting servers.| in E:\D\PTNP64b\PlusLib\src\PlusServer\Tools\PlusServer.cxx(116)
071423_192355.634|WARNING|001.137000| Buffer item is not in the buffer (Uid: 0)!| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusTimestampedCircularBuffer.cxx(176)
071423_192355.634|WARNING|001.137000| Unable to get timestamp from Marker0ToTracker tool tracker buffer for time: 0| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(1295)
071423_192355.634|WARNING|001.137000| Buffer item is not in the buffer (Uid: 0)!| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusTimestampedCircularBuffer.cxx(176)
071423_192355.635|WARNING|001.138000| Unable to get timestamp from Marker1ToTracker tool tracker buffer for time: 0| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(1295)
071423_192355.635|ERROR|001.138000| Failed to get most recent timestamp from all the tracker tools| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(1312)
071423_192355.638|INFO|001.141000| Plus OpenIGTLink server listening on IPs: 192.168.17.88, 169.254.14.106, 169.254.63.16, 127.0.0.1, 172.23.112.1, 172.18.224.1 – port 18944| in E:\D\PTNP64b\PlusLib\src\PlusServer\vtkPlusOpenIGTLinkServerWin32.cxx(81)
071423_192355.692|INFO|001.195000| Plus OpenIGTLink server listening on IPs: 192.168.17.88, 169.254.14.106, 169.254.63.16, 127.0.0.1, 172.23.112.1, 172.18.224.1 – port 18945| in E:\D\PTNP64b\PlusLib\src\PlusServer\vtkPlusOpenIGTLinkServerWin32.cxx(81)
071423_192355.735|INFO|001.238000| Server status: Server(s) are running.| in E:\D\PTNP64b\PlusLib\src\PlusServer\Tools\PlusServer.cxx(147)
071423_192355.735|INFO|001.238000| Press Ctrl-C to quit.| in E:\D\PTNP64b\PlusLib\src\PlusServer\Tools\PlusServer.cxx(148)

The message window of Plus server saying

Connection successful!

Tracking of printed ArUco markers using a simple camera. Marker positions and image data are broadcasted through OpenIGTLink (on port 18944 and 18945, respectively). To use a different camera, change CaptureDeviceId attribute (to 0, 1, 2, …).

Plus OpenIGTLink server listening on IPs: 192.168.17.88, 169.254.14.106, 169.254.63.16, 127.0.0.1, 172.23.112.1, 172.18.224.1 – port 18944
Plus OpenIGTLink server listening on IPs: 192.168.17.88, 169.254.14.106, 169.254.63.16, 127.0.0.1, 172.23.112.1, 172.18.224.1 – port 18945

Then I switched to OpenIGTLinkIF module, and change the port number from 18944 to 18945.
After that, I turned the “Active” checkbox on. I showed Aruco marker printed from the slide to my webcam, however, I couldn’t find translation node ("marker0totracker or "marker1totracker) in"I/O configulation window.

capture2
capture21920Ă—1080 256 KB

the Plus log file in this situation is

071423_193351.080|ERROR|009.708000| Pose estimation failed. Tool Marker0ToTracker with marker 0.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)
071423_193351.082|ERROR|009.709000| Pose estimation failed. Tool Marker1ToTracker with marker 1.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)
071423_193351.115|ERROR|009.742000| Pose estimation failed. Tool Marker0ToTracker with marker 0.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)
071423_193351.115|ERROR|009.742000| Pose estimation failed. Tool Marker1ToTracker with marker 1.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)
071423_193351.151|ERROR|009.778000| Pose estimation failed. Tool Marker0ToTracker with marker 0.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)
071423_193351.151|ERROR|009.778000| Pose estimation failed. Tool Marker1ToTracker with marker 1.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)
071423_193351.184|ERROR|009.812000| Pose estimation failed. Tool Marker1ToTracker with marker 1.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)
071423_193351.215|ERROR|009.843000| Pose estimation failed. Tool Marker1ToTracker with marker 1.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)
071423_193351.253|ERROR|009.880000| Pose estimation failed. Tool Marker1ToTracker with marker 1.| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpticalMarkerTracking\vtkPlusOpticalMarkerTracker.cxx(416)

I appriciate your support.

4

I think I could solved this by my self after the latest reply…

When I created two scene, one is listening port 18944, another is listening 18945,
pointer model apperead and which seemed to move with the marker moving in front of the Webcam.

capture5
capture51903Ă—1043 233 KB

It is little bit lagged, even with low pass filter provided by transform processor module. But it’s fine.

I am still wondering I solved this or not. But your kind reply encourage me to face this.

1 Like
5

Yeah I think you’ve solved it. The config file has 2 OpenIGTLink connections. The one on 18944 is transmitting the marker transforms, and the one on 18945 is transmitting the webcam image.

1 Like
6

Hi, I am new to 3D Slicer, can show the step how to make the Aruco Marker tracking work as the Video exactly? I have followed the tutorial and made webcam function, created the needle model, however the tracking position and direction is not I expected.

https://www.youtube.com/watch?v=MOqh6wgOOYs