# Live image transfer via TCP/IP

**URL:** <https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866>\
**Category:** Development\
**Tags:** augmented-reality\
**Created:** [November 26, 2018, 2:22am UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866 "2018-11-26T02:22:08Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![shellyverde](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/shellyverde/32/1186_2.png) [@shellyverde](https://discourse.slicer.org/u/shellyverde)\
**Post date:** [November 26, 2018, 2:22am UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/1 "2018-11-26T02:22:08Z")

</div>

Operating system: Windows  
Slicer version: 4.8

Hi,

I need to connect a device with Slicer through TCP/IP socket connection. The device runs in real time and the data from the device need to be sent to Slicer in every certain milliseconds. Meanwhile commands will send to the device through Slicer. Could you please suggest how to implement the TCP/IP communication? Can I use OpenIGTLink? If yes, could I find any documents or examples? Shall I implement the socket connection using Qt Socket, Qt TCPServer, etc.? Or any other way?  
Any suggestion is highly appreciated!

-Shelly

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [November 26, 2018, 5:07am UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/2 "2018-11-26T05:07:48Z")

</div>

OpenIGTLink is a very simple TCP/IP protocol that is already well supported in Slicer. I would highly recommend using it, as it is not easy to implement stable and efficient multi-threaded communication in an frontend application. You can find OpenIGTLink implementations for most common programming languages (C++, Python, Java, MATLAB, etc.).

---

<div class="post-metadata">

**Author:** ![shellyverde](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/shellyverde/32/1186_2.png) [@shellyverde](https://discourse.slicer.org/u/shellyverde)\
**Post date:** [November 27, 2018, 3:45am UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/3 "2018-11-27T03:45:38Z")

</div>

Thanks! Will try it.  
-Shelly

---

<div class="post-metadata">

**Author:** ![slicer365](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/slicer365/32/67549_2.png) [@slicer365](https://discourse.slicer.org/u/slicer365)\
**Post date:** [July 22, 2023, 4:25pm UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/4 "2023-07-22T16:25:02Z")

</div>

Sorry to disturb you ,Mr Lasso！  
I wrote a server script in python to get the coordinates sent from slicer, but what I got was bytes, how to convert it into a string? Decode is still error.

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/2/0/20caeca9ca1d0750fcbfab1ea63cd0fab3e58007.png)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 22, 2023, 4:38pm UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/5 "2023-07-22T16:38:49Z")

</div>

There is no need to develop a Python script for this, because sending/receiveing transforms, markups, images, etc. to/from Slicer via TCP/IP (using OpenIGTLink protocol) is already implemented in [pyigtl](https://pypi.org/project/pyigtl/) Python package.

---

<div class="post-metadata">

**Author:** ![slicer365](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/slicer365/32/67549_2.png) [@slicer365](https://discourse.slicer.org/u/slicer365)\
**Post date:** [July 22, 2023, 5:09pm UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/6 "2023-07-22T17:09:44Z")

</div>

Thank you!  
If I need to get the coordinates sent by slicer in real time, or when the coordinates in slicer change, it will trigger the sending coordinates, how should I write this script?

Is this right？

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/b/1/b13286c6a2117fe20c7e9c7bbf5fa0352a80a0f5.png)

and If I want send a transfrom from Unity Scene to slicer by IGT，how should I define the transform.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 22, 2023, 10:22pm UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/7 "2023-07-22T22:22:14Z")

</div>

Yes, Slicer sends the updated transform, image, markup, etc. automatically when there is a change. You can set it up in OpenIGTLinkIF module.

If you want to use Unity then I would recommend to start from an existing project, there are a couple of them that show the Slicer scene, allows moving nodes by grabbing and moving them physically, reslice images, etc. Search for augmented reality and HoloLens in this forum for more information.

---

<div class="post-metadata">

**Author:** ![slicer365](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/slicer365/32/67549_2.png) [@slicer365](https://discourse.slicer.org/u/slicer365)\
**Post date:** [July 23, 2023, 1:57am UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/8 "2023-07-23T01:57:17Z")

</div>

Thank you!  
In addition，I used the example script, it runs normally.

How can I receive the position message in slicer

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/5/0/50a8a06c38719a483aa67f73a810177490810b3c.png)

There is no message in the slicer.

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/b/f/bf8e7f88523280ce315a21731e05661e7834d99b.png)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 23, 2023, 4:42am UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/9 "2023-07-23T04:42:05Z")

</div>

It did not make sense to add the POSITION message to Slicer, as it just barely smaller than a TRANSFORM message and way more limited. I would recommend to use the TRANSFORM message instead.

---

<div class="post-metadata">

**Author:** ![slicer365](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/slicer365/32/67549_2.png) [@slicer365](https://discourse.slicer.org/u/slicer365)\
**Post date:** [July 24, 2023, 4:34pm UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/10 "2023-07-24T16:34:25Z")

</div>

Dear Mr Lasso，

I want to receive the image from slicer through Python and display it. My code is like this. I can get the message, but how can I filter out the array and display it as a picture?

Thank you ！

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/f/9/f938d69c90166e3e630c98586f2ece63a789caf0.jpeg)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 24, 2023, 4:38pm UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/11 "2023-07-24T16:38:01Z")

</div>

It seems that you are considering reimplementing some image visualization, which should not be necessary. What is your overall goal?

---

<div class="post-metadata">

**Author:** ![slicer365](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/slicer365/32/67549_2.png) [@slicer365](https://discourse.slicer.org/u/slicer365)\
**Post date:** [July 24, 2023, 4:43pm UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/12 "2023-07-24T16:43:33Z")

</div>

I want to transfer the image in slicer to other software in real time.  
For example, I develop a mobile phone software based on TCP, and then I can operate it on my computer and transmit the image to my phone in real time or Hololens device.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 24, 2023, 5:45pm UTC](https://discourse.slicer.org/t/live-image-transfer-via-tcp-ip/4866/13 "2023-07-24T17:45:29Z")

</div>

For HoloLens there are already full-feature open-source augmented reality applications that you can use as is, or customize/extend to match your needs. See for example:

> **[GitHub - BSEL-UC3M/HoloLens2and3DSlicer-PedicleScrewPlacementPlanning: GitHub...](https://github.com/BSEL-UC3M/HoloLens2and3DSlicer-PedicleScrewPlacementPlanning)**
>
> GitHub repository for the IJCARS paper: "Real-Time open-source integration between Microsoft HoloLens 2 and 3D Slicer" - GitHub - BSEL-UC3M/HoloLens2and3DSlicer-PedicleScrewPlacementPlann...