Learning how to use pyigtl

Support
, ,
1

Operating system: 10
Slicer version: n/a
Expected behavior: receiving an imaged with embedded transform
Actual behavior: received “None”

hi all,

I am trying to create a python program that receives tracked images from PlusLib via pyigtl. I am trying to understand how to use pyigtl effectively and am seeking your help in understanding it.

First, I downloaded the latest stable version of Plus Toolkit. Using a simple webcam as a tracker, I launched the Plus Server Launched and used “PlusServer: Optical Marker Tracker using MMF video”. Using the following code, I am able to receive tracking transform of an Aruco Marker:

import pyigtl
tclient = pyigtl.OpenIGTLinkClient("127.0.0.1", 18944)
tmessage = tclient.wait_for_message("Marker0ToTracker", timeout=5)
print(tmessage)

However, modifying the above to acquire the tracked image always returns with a message of “None”:

import pyigtl
client = pyigtl.OpenIGTLinkClient("127.0.0.1", 18945)
message = client.wait_for_message("Image", timeout=5)
print(message)

suggesting that the tracked image is not being received. Please note that the image is transmitted via a different port than the transforms themselves.

can anyone suggest a reason why the tracked images are not been received?

for complete reference, the plus server XML is listed below:

<PlusConfiguration version="2.4">
  <DataCollection StartupDelaySec="1.0" >
    <DeviceSet
      Name="PlusServer: Optical marker tracker using MMF video"
      Description="Tracking of printed ArUco markers using a simple camera. Marker positions and image data are broadcasted through OpenIGTLink (on port 18944 and 18945, respectively). To use a different camera, change CaptureDeviceId attribute (to 0, 1, 2, ...)." />
    <Device
      Id="VideoDevice"
      Type="MmfVideo"
      FrameSize="640 480"
      VideoFormat="YUY2"
      CaptureDeviceId="1" >
      <DataSources>
        <DataSource Type="Video" Id="Video" PortUsImageOrientation="MF" ImageType="RGB_COLOR" />
      </DataSources>
      <OutputChannels>
        <OutputChannel Id="VideoStream" VideoDataSourceId="Video" />
      </OutputChannels>
    </Device>
    <Device
      Id="TrackerDevice"
      Type="OpticalMarkerTracker"
      CameraCalibrationFile="OpticalMarkerTracker/realsense_gen2_calibration.yml"
      ToolReferenceFrame="Tracker"
      TrackingMethod="OPTICAL"
      MarkerDictionary="ARUCO_MIP_36h12">
      <DataSources>
        <DataSource Type="Tool" Id="Marker0" MarkerId="0" MarkerSizeMm="80" />
        <DataSource Type="Tool" Id="Marker1" MarkerId="1" MarkerSizeMm="80" />
        <DataSource Type="Tool" Id="Marker2" MarkerId="2" MarkerSizeMm="50" />
        <DataSource Type="Tool" Id="Marker3" MarkerId="3" MarkerSizeMm="50" />
        <DataSource Type="Tool" Id="Marker4" MarkerId="4" MarkerSizeMm="50" />
        <DataSource Type="Tool" Id="Marker5" MarkerId="5" MarkerSizeMm="30" />
        <DataSource Type="Tool" Id="Marker6" MarkerId="6" MarkerSizeMm="30" />
        <DataSource Type="Tool" Id="Marker7" MarkerId="7" MarkerSizeMm="30" />
        <DataSource Type="Tool" Id="Marker8" MarkerId="8" MarkerSizeMm="30" />
      </DataSources>
      <InputChannels>
        <InputChannel Id="VideoStream"  />
      </InputChannels>
      <OutputChannels>
        <OutputChannel Id="TrackerStream" >
          <DataSource Id="Marker0"/>
          <DataSource Id="Marker1"/>
          <DataSource Id="Marker2"/>
          <DataSource Id="Marker3"/>
          <DataSource Id="Marker4"/>
          <DataSource Id="Marker5"/>
          <DataSource Id="Marker6"/>
          <DataSource Id="Marker7"/>
          <DataSource Id="Marker8"/>
        </OutputChannel>
      </OutputChannels>
    </Device>
    <Device
      Id="CaptureDevice"
      Type="VirtualCapture"
      BaseFilename="RecordingTest.igs.mha"
      EnableCapturingOnStart="FALSE" >
      <InputChannels>
        <InputChannel Id="TrackerStream" />
      </InputChannels>
    </Device>
  </DataCollection>
  
  <PlusOpenIGTLinkServer
    MaxNumberOfIgtlMessagesToSend="1"
    MaxTimeSpentWithProcessingMs="50"
    ListeningPort="18944"
    SendValidTransformsOnly="true"
    OutputChannelId="TrackerStream" >
    <DefaultClientInfo>
      <MessageTypes>
        <Message Type="TRANSFORM" />
      </MessageTypes>
      <TransformNames>
        <Transform Name="Marker0ToTracker" />
        <Transform Name="Marker1ToTracker" />
        <Transform Name="Marker2ToTracker" />
        <Transform Name="Marker3ToTracker" />
        <Transform Name="Marker4ToTracker" />
        <Transform Name="Marker5ToTracker" />
        <Transform Name="Marker6ToTracker" />
        <Transform Name="Marker7ToTracker" />
        <Transform Name="Marker8ToTracker" />
      </TransformNames>
    </DefaultClientInfo>
  </PlusOpenIGTLinkServer>

  <PlusOpenIGTLinkServer 
    MaxNumberOfIgtlMessagesToSend="1" 
    MaxTimeSpentWithProcessingMs="50" 
    ListeningPort="18945" 
    SendValidTransformsOnly="true" 
    OutputChannelId="VideoStream" > 
    <DefaultClientInfo> 
      <MessageTypes> 
        <Message Type="IMAGE" />
      </MessageTypes>
      <ImageNames>
        <Image Name="Image" EmbeddedTransformToFrame="Image" />
      </ImageNames>
    </DefaultClientInfo>
  </PlusOpenIGTLinkServer>

</PlusConfiguration>
2

Can Slicer receive a continuous stream of images from this server?

If yes, then the issue is in pyigtl. Since it is a pure native Python package, you can debug it very easily in any Python IDE. Please step through the code using a debugger and let us know where things go wrong.

3

hi Andras,

thanks for the hint. Indeed it works in Slicer. The mistake I made was that I should be querying “Image_Image” instead of just “Image” in the code above. Now it works just fine.

regards,

1 Like