Unexpected behavior in Transforms Module and IGT ResliceDriver

Development
,
1

When tracking two instruments using the Brainlab system and OpenIGTLink:

  1. OpenIGT link transfers transforms for both instruments from BrainLab to Slicer as expected
  2. As long as only one instrument is ever visible at a time, things work as expected
    a) Transform sliders and transforms change in real time in the Transform Module as expected
    b) Slice views are controlled by the transforms assigned in the ResliceDriver as expected
  3. If at some point, both instruments become visible, the Transforms Module and ResliceDriver both behave in unexpected ways
    a) As long as both instruments are visible and until one of the instruments becomes not visible:
    • Transform sliders and transforms behave as expected in the Transforms Module
    • Slice views are controlled as expected by the ResliceDriver
      b) As soon as one of the two instruments becomes not visible and thereafter until Slicer is stopped and restarted:
    • The transform from one of the two instruments (always the same one, possibly determined by the order Brainlab sends transforms) no longer impacts the Transforms Module (sliders and transforms do not change when the instrument is moved) and the ResliceDriver (transform does not control the assigned slice view)
      c) Note that both transforms continue to be received via OpenIGTLink as expected and both emit move events. (We have an independent module that observes and records transform events. The behavior of the Transforms Module and ResliceDriver described here occurs whether or not our module is opened/activated.)

I think this may have something to do with how observers are attached/detached from transforms in the Transforms Module and ResliceDriver. However, the behavior is buried under multiple layers in vtk and MRML that I am having trouble tracing through the code.

Is there anyone familiar with these modules and/or the vtk observer model who might be able to help us with this issue? It is a blocking issue on software we used regularly during tumor resections.

Thanks!
Sarah

2

Hi Sarah,

It sounds very strange that your custom module can successfully observe and record the transforms, but the Transforms module cannot.

Is the MRML transform node representing the problematic instrument updated in any way when the instrument is in view of the tracker? Is there maybe another MRML node in the scene created for some reason and that is being updated?

Each MRML node has an “MTime” property that you can watch using the Node info module. That is a timestamp for when it was last modified. Is that changing while you experience the problem? If that is not changing, then most modules would not get notified about the updates. (But it should change when new messages are received from the tracker.)

When IGTL transform messages arrive in Slicer, they are converted to MRML nodes, identified by the IGTL device name and the matching MRML node name. If there IGTL device name changes, then there may be no matching MRML node, so OpenIGTLinkIF creates one that is not associated with the reslice driver. Any chance the IGTL device names changes while the problem is happening?

These are just some ideas to start thinking along. Please let me know if you figure out more.

Tamas

3

Hi Tamas, thanks very much for your quick reply. I will do some more sleuthing based on your suggestions. In the meantime, here are some things we’ve observed:

It sounds very strange that your custom module can successfully observe and record the transforms, but the Transforms module cannot.

Yes, we agree it is odd. We checked this several times to make sure.

Is the MRML transform node representing the problematic instrument updated in any way when the instrument is in view of the tracker? Is there maybe another MRML node in the scene created for some reason and that is being updated?

This is possible. We checked that there is only one transform per instrument listed in the Data Module. I can check to see if there are some unnamed nodes that don’t appear there.

Each MRML node has an “MTime” property that you can watch using the Node info module. That is a timestamp for when it was last modified. Is that changing while you experience the problem? If that is not changing, then most modules would not get notified about the updates. (But it should change when new messages are received from the tracker.)

When I record transforms in my module (which sees both instruments changing even when Transforms and ResliceDriver do not), I also record the times of the transform matrix – these do change with each event. I can double check in the Transforms Module to see if they change there.

When IGTL transform messages arrive in Slicer, they are converted to MRML nodes, identified by the IGTL device name and the matching MRML node name. If there IGTL device name changes, then there may be no matching MRML node, so OpenIGTLinkIF creates one that is not associated with the reslice driver. Any chance the IGTL device names changes while the problem is happening?

I can dig deeper into this, possibly by printing out all the transform nodes to see if they change when the behavior occurs. Perhaps my module identifies the transform with a field that doesn’t change while the other Slicer modules use one that does? I will see what I can discover.

I will try these next time I have access to the equipment (probably next week). Let me know if you think of other things I should test.

Sarah

4

Hi Sarah,

Did this configuration work correctly before? Or is this a new setup that you’d like to get working? If it used to work but stopped working, it could be useful to know if there was any update on any side after which this problem started.

Do you see any errors in the Python console? An obvious question but could help a lot, and I don’t see it mentioned that there are no apparent error messages.

I’m not familiar with how BrainLab is connected to Slicer, but is PLUS involved? If so, can you share the configuration XML file please?

5

Hi Csaba, thanks for thinking about this for us. The clinical team assures us that the configuration worked a few years ago. It is not clear exactly when it broke – the earliest I heard about it was about 1 year ago. I only started digging into it this month. It is possible that Brainlab updated their software during that period but we don’t have access to that information.

I don’t see any errors in the Python code. I have my Python logging level set to ‘info’. We also checked the log and didn’t see anything obvious.

We don’t use Plus, so there is no config file. We use OpenIGTLinkIF to connect to Brainlab and OpenIGTLinkRemote to access tracking data (using the Start/Stop buttons).

The strange thing is that my custom module gets the expected transform events but the Transforms Module and ResliceDriver do not seem to get them consistently (see below) This happens even when my module is not installed. I wonder if it has to do with event processing. In my module, I use AddObserver() in my tracked instrument class to observe and respond to transform changes, while the two Slicer modules use setAndObserveTransformNodeID(). I tried to see how these approaches are related, but got lost in the class layers.

I’m just speculating. However, what if Brainlab or OpenIGTLink changed from sending an event per transform to sending a bundle of transforms per event? It might cause unexpected behavior if more than one transform is sent and only the first transform is processed.

For a recap of the behavior: assuming we are tracking two instruments, i1 and i2 then

  1. If only one instrument (i1 or i2) is ever visible at a time, the Transforms Module and the Reslice Driver observe changes to both transforms.

  2. If at any point, both i1 and i2 become visible at the same time, things chage. Thereafter
    a) if both are visible and both are moving, the two slicer modules respond to the transform of i1 but do not respond to the transform of i2 unless i1 is stationary
    b) if i2 is visible but i1 is not visible, the two slicer modules do not respond to the transform of i2

I have access to the Brainlab system this Friday (we have limited access because it’s used clinically) so I’m going to make the tests Tomas suggested. Let me know if there is anything else you think I should try.

Many thanks!
Sarah

6

Thank you for the detailed answer! I have a very hard time imagining that OpenIGTLink has observation issues. It has been working (I’d guess) for at least a decade without any considerable changes to the base code, and it can handle more than two tracked devices without an issue. I see the probability of this being a bug in OpenIGTLink really low.

The only thing that keeps popping into my head (without knowing anything on how the BrainLab connection is set up, but surely there is some kind of configuration there) is that the name or ID of the two transforms are the same, or something of this sort, and one “overwrites” the other, and causes some kind of conflict.

Sorry I know this is very vague. If you can share more details about the connection to the hardware after your visit on Friday maybe we can try to diagnose the issue better.