# Streaming data from a program to a Slicer extension

**URL:** <https://discourse.slicer.org/t/streaming-data-from-a-program-to-a-slicer-extension/23952>\
**Category:** Development\
**Created:** [June 19, 2022, 11:05pm UTC](https://discourse.slicer.org/t/streaming-data-from-a-program-to-a-slicer-extension/23952 "2022-06-19T23:05:18Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![koeglfryderyk](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/koeglfryderyk/32/14638_2.png) [@koeglfryderyk](https://discourse.slicer.org/u/koeglfryderyk)\
**Post date:** [June 19, 2022, 11:05pm UTC](https://discourse.slicer.org/t/streaming-data-from-a-program-to-a-slicer-extension/23952/1 "2022-06-19T23:05:18Z")

</div>

I have a Python program that detects hand coordinates in a video stream.

I’m trying to send those coordinates continuously to my Slicer Python extension so I can display the hand in Slicer.

I think I’m half-way there. I can update the hand model in Slicer by repeatedly pressing a button. This is what I currently have:

1. The python program continuously sends data using `from multiprocessing.connection import Client`, it looks more or less like this:

```auto
with Client(('localhost', 5038)) as c:
    msg = np.reshape(all_fings_world[:, 0:3], (63, 1)).tobytes()
    c.send(msg)

```

2. In Slicer, in my extension I created a button that when clicked executes code that looks more or less like this:

```auto
try:
    self.serv = Listener(('', 5038))
    client = self.serv.accept()
    self.message = client.recv()
    self.update_hand()
except Exception as e:
    print(e)

```

What I’m missing is to not have to keep clicking this button to update the hand in Slicer. Is there a way to make Slicer execute some code in a loop in the ‘background’ so that Slicer can still be used? I tried creating a thread to run this code, but this didn’t do anything. And just putting everything in a `while True` freezes Slicer.

---

<div class="post-metadata">

**Author:** ![errollgarner](https://avatars.discourse-cdn.com/v4/letter/e/8e8cbc/32.png) [@errollgarner](https://discourse.slicer.org/u/errollgarner)\
**Post date:** [June 20, 2022, 1:02am UTC](https://discourse.slicer.org/t/streaming-data-from-a-program-to-a-slicer-extension/23952/2 "2022-06-20T01:02:24Z")

</div>

You can use OpenIGTLink to send and receive the points:  
In your python program, you can use the python implementation of OpenIGTLink ([GitHub - lassoan/pyigtl: Python implementation of OpenIGTLink](https://github.com/lassoan/pyigtl)) to send and on the Slicer side you can use the OpenIGTLinkIF module to receive the data.  
OpenIGTLink natively supports streaming points. You can find the protocol specifications here: [OpenIGTLink/point.md at release-3.0 · openigtlink/OpenIGTLink · GitHub](https://github.com/openigtlink/OpenIGTLink/blob/release-3.0/Documents/Protocol/point.md)

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [June 20, 2022, 7:56pm UTC](https://discourse.slicer.org/t/streaming-data-from-a-program-to-a-slicer-extension/23952/3 "2022-06-20T19:56:33Z")

</div>

OpenIGTLink is a good option. Another option would be to use the new [WebServer](https://slicer.readthedocs.io/en/latest/user_guide/modules/webserver.html?highlight=webserver#data-access) to control the scene. It is also integrated in the Slicer event loop so it handles updates asynchronously.

Or if you want to stick with something lower level like you are currently doing you could use a `QSocketNotifier` like is done [in the WebServer code](https://github.com/Slicer/Slicer/blob/master/Modules/Scripted/WebServer/WebServer.py#L487) so that you get a Qt signal when the socket has data ready to read.

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [June 21, 2022, 1:55am UTC](https://discourse.slicer.org/t/streaming-data-from-a-program-to-a-slicer-extension/23952/4 "2022-06-21T01:55:54Z")

</div>

> [@pieper](#):
>
> ould be to use the new [WebServer](https://slicer.readthedocs.io/en/latest/user_guide/modules/webserver.html?highlight=webserver#data-access) to control the scene

Is this in latest stable? I am not seeing a module called webserver.

---

<div class="post-metadata">

**Author:** ![jamesobutler](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jamesobutler/32/7511_2.png) [@jamesobutler](https://discourse.slicer.org/u/jamesobutler)\
**Post date:** [June 21, 2022, 3:27am UTC](https://discourse.slicer.org/t/streaming-data-from-a-program-to-a-slicer-extension/23952/5 "2022-06-21T03:27:32Z")

</div>

WebServer was integrated into Slicer on May 11th or about a week after the Slicer 5.0.2 release. Therefore it is only available in Slicer Preview builds since then.

> <https://github.com/Slicer/Slicer/commit/bdc1ee03893e957ba360a5988e4ed93cc275637a>
>
> \* ENH: add WebServer module
> 
> Code imported from Steve Pieper's developments.
> …
> https://github.com/pieper/SlicerWeb
> 
> \* DOC: add WebServer documentation page
> 
> \* DOC: add note about web server access
> 
> \* ENH: Improve WebServer module
> 
> \- Moved out request handlers to separate files.
> \- Allow registration of custom request handlers.
> \- Iterate through registered request handlers and let that one process the request that returned the highest confidence.
> \- Save default handler and logging options in application settings.
> \- Start/stop the server when a start/stop button is pressed (not start when opening the WebServer module GUI).
> \- Updated Slicer icon.
> \- Updated module icon (API icon from Google Material Icon Set)
> \- Added \`sampleData\` command to allow easy testing and demoing of the API
> \- Added \`gui\` command to switch between full application/viewers only, switch viewer layout
> \- Added \`screenshot\` command to get screenshot of the application main window
> 
> \* DOC: Update webserver.md
> 
> \* ENH: web server fixes
> 
> \* add entry on demo page to reset view to four up with controls
> \* change "got it" messages to more meaningful feedback
> \* add extra tooltip hints on usage
> \* fix logic of local connection button
> 
> \* DOC: add web server security suggestion
> 
> \* STYLE: fix codespell findings
> 
> https://github.com/Slicer/Slicer/actions/runs/2254569488
> 
> \* ENH: add exception handler for web requests
> 
> This puts the stack trace in the log and prevents
> any bad request handlers from leaving the network connection
> dangling.
> 
> \* BUG: fix dicomweb in Web Server module
> 
> A newer version of pydicom is required to fix a bug
> in json serialization of a pydicom multivalue class.
> 
> Also fix a warning from bad uri syntax.
> 
> \* BUG: fix DICOMweb endpoint for OHIF
> 
> \* Implements study level query and metadata.
> \* Adds offset and limits to study listing
> \* Adds documentation on using with OHIF
> 
> Co-authored-by: Andras Lasso \<lasso@queensu.ca\>