I am trying to receive some data via OpenIGTLink with additional data added. For example I can already receive a vtkMRMLScalarVolumeNode but in this node I cannot include information about the patient (e.g. UID). In the overview page of OpenIGTLinkIF I found some information that BindMessages could solve the problem (https://www.slicer.org/wiki/Documentation/Nightly/Developers/OpenIGTLinkIF).
Unfortunately I have not found further information on how to use it and in the source files of the project I also hardly found any hints. Are BindMessages support via the Python-Interface at all?
Anyone has a small example on how to use them?
Could you write a bit more about your use case? There can be many solutions and we would need to know what your preferences and constraints are. What kind of images do you send? How often? How often do metadata change?
You can already send patient metadata in a STRING message when you start a procedure. @Sunderlandkyl’s proposal of saving metadata as node attribute would be useful in general and may help in your case, too.
Thanks for your reply. I almost thought that there is currently no such feature. But now I know for sure and can stop searching, thanks.
I am not sure whether your suggestion would help us in this case. We have a server running on another machine which processes data and then sends results (e.g. segmentation image) to our client. On client side we need to identify to which volume series the result belongs to. The server could send a message (Text) before or after the actual result for communicating the corresponding UID. But since the server is multi-threaded, results for different datasets could potentially arrive randomly.
Adding such info into somewhere in the metadata of the ogtl::ImageMessage is not possible, is it? Or is this what you are suggesting?
the use case I have just described partially in my last reply. Maybe you are right and it will be sufficient to send such data only if the currently processed patient is changing. On both machines we received new volume data via dicom nodes which shall be processed on the fly. Our client allows to load and view the data in Slicer, the server does some post-processing based on the volume and sends the results. We need to make sure that each result is assigned to the correct acquisition. But we can assume that new acquisitions are not made too often. So probably we simply send it once.
You can identify volumes in Slicer based on the “device name” OpenIGTLink message field. This field is used as volume node name when Slicer receives the message. You can use any string, such as a randomly generated ID or by concatenation patient/study/series IDs. OpenIGTLink 2 has a limit of 20 characters but I think OpenIGTLink 3 has a mechanism to use longer device name.
I am wondering if the solution of allowing node attributes to be sent as image metadata has been implemented?
My use case is that I have an AI inference listening to a pyigtl server. The AI model needs to know not just the pixel values of the image but other metadata from the image as well - specifically it needs to know the physical height and width of the image so it can select the appropriate ROIs to make predictions.
I think allowing node attributes to go through the OpenIGTLink as image metadata would be a good solution. My current workaround is to send text messages with serialized attribute dictionaries through the interface, but it is not the cleanest.
Physical height and width is already included in the basic IMAGE message (without that Slicer could not correctly display an image). Note that to define the image geometry, specifying spacing is not sufficient, but you also need to specify origin, spacing, axis directions.
You can also add any additional metadata to any message (in the metadata property of the message object in pyigtl) and I believe all metadata fields appear as MRML node attributes in Slicer.
“Physical height and width is already included in the basic IMAGE message (without that Slicer could not correctly display an image). Note that to define the image geometry, specifying spacing is not sufficient, but you also need to specify origin, spacing, axis directions.”
I did notice that the ijk to world matrix is included in the image message. However, I have not been using the ijk to world matrices in my slicer module but instead i’ve been using a transform to put the image into the correct coordinate system. I think Tamas told me it would be better to use a transform rather than touch the ijk to world matrices. Therefore the image message that OpenIGTLinkIF is sending to my python process is not getting the info about the image geometry that it needs. I guess I could send the transform that it is observing as another message to pyigtl.
“You can also add any additional metadata to any message (in the metadata property of the message object in pyigtl) and I believe all metadata fields appear as MRML node attributes in Slicer.”
This is a useful feature, but in my case I need to send metadata attributes from OpenIGTLinkIF to pyigtl, rather than the other way around.
It is more flexible if you send the image in the IJK coordinate system and the IJKTo?Transform and other transforms in separate messages. This flexibility is useful if you build a complex surgical guidance system with live images, various tracked tools, etc.
If all you need to send is an image in physical space then embedding origin, spacing, axis directions in the IMAGE message may be simpler.
Slicer only sends a few selected MRML node attributes as OpenIGTLink message metadata (e.g., transform status). It wouldn’t be hard to generalize this (e.g., allow specifying a list of MRML node attribute names that would be included as metadata).
Sounds good, I’ll use one of those two options. Do you have any examples of programs that use OpenIGTLinkIF for communicating with an AI server? Also, do you have any examples of Scripted modules that access the OpenIGTLinkIF logic to programatically set up connections?