Streaming data from a program to a Slicer extension

Development
1

I have a Python program that detects hand coordinates in a video stream.

I’m trying to send those coordinates continuously to my Slicer Python extension so I can display the hand in Slicer.

I think I’m half-way there. I can update the hand model in Slicer by repeatedly pressing a button. This is what I currently have:

  1. The python program continuously sends data using from multiprocessing.connection import Client, it looks more or less like this:
    with Client(('localhost', 5038)) as c:
    msg = np.reshape(all_fings_world[:, 0:3], (63, 1)).tobytes()
    c.send(msg)
  2. In Slicer, in my extension I created a button that when clicked executes code that looks more or less like this:
    try:
    self.serv = Listener(('', 5038))
    client = self.serv.accept()
    self.message = client.recv()
    self.update_hand()
except Exception as e:
    print(e)

What I’m missing is to not have to keep clicking this button to update the hand in Slicer. Is there a way to make Slicer execute some code in a loop in the ‘background’ so that Slicer can still be used? I tried creating a thread to run this code, but this didn’t do anything. And just putting everything in a while True freezes Slicer.

2

You can use OpenIGTLink to send and receive the points:
In your python program, you can use the python implementation of OpenIGTLink (GitHub - lassoan/pyigtl: Python implementation of OpenIGTLink) to send and on the Slicer side you can use the OpenIGTLinkIF module to receive the data.
OpenIGTLink natively supports streaming points. You can find the protocol specifications here: OpenIGTLink/point.md at release-3.0 · openigtlink/OpenIGTLink · GitHub

2 Likes
3

OpenIGTLink is a good option. Another option would be to use the new WebServer to control the scene. It is also integrated in the Slicer event loop so it handles updates asynchronously.

Or if you want to stick with something lower level like you are currently doing you could use a QSocketNotifier like is done in the WebServer code so that you get a Qt signal when the socket has data ready to read.

1 Like
4

Is this in latest stable? I am not seeing a module called webserver.

5

WebServer was integrated into Slicer on May 11th or about a week after the Slicer 5.0.2 release. Therefore it is only available in Slicer Preview builds since then.

1 Like