Operating system: Ubuntu 16.04 LTS
Slicer version: 4.7.0-2016-11-08 r25516
Expected behavior: Real time ultrasound images display.
Actual behavior: Slicer crashes when the Volumes module is selected.
Hi,
I have an ultrasound scanner connected to the computer on which Slicer runs. I use a TCP/IP protocol to send and retrieve images generated by the US scanner. A client runs on the US scanner and a QTcpServer runs on Slicer. On the first connection of the US scanner the following code is executed:
//Setup image importer
//_image is just a QByteArray
_image.resize(800);//the first image to be displayed has an arbitrary size
_image.fill(0);
//_imageImport is a smart pointer on a vtkImageImport object
_imageImport->SetDataSpacing(1, 1, 1);
_imageImport->SetDataOrigin(0, 0, 0);
_imageImport->SetWholeExtent(0, 9, 0, 9, 0, 0);
_imageImport->SetDataExtentToWholeExtent();
_imageImport->SetDataScalarTypeToDouble();
_imageImport->SetNumberOfScalarComponents(1);
_imageImport->SetImportVoidPointer(_image.data());
_imageImport->Update();
//Setup and add display node to the scene
_bModeDisplayNode->SetScene(mrmlScene());
_bModeDisplayNode->SetAutoWindowLevel(0);
_bModeDisplayNode->SetWindowLevelMinMax(0, 64);
_bModeDisplayNode->SetInterpolate(0);
_bModeDisplayNode->SetAndObserveColorNodeID("vtkMRMLColorTableNodeGrey");
mrmlScene()->AddNode(_bModeDisplayNode);
//Setup volume node
_bModeVolumeNode->SetScene(mrmlScene());
_bModeVolumeNode->SetAndObserveImageData(_imageImport->GetOutput());
_bModeVolumeNode->SetAndObserveDisplayNodeID(_bModeDisplayNode->GetID());
mrmlScene()->AddNode(_bModeVolumeNode);
Then every time a new image is ready to be displayed, the following code is executed:
_imageImport->SetWholeExtent(0, _imageDepth-1, 0, _imageWidth-1, 0, 0);
_imageImport->SetDataExtentToWholeExtent();
_imageImport->SetImportVoidPointer(_image.data());
_imageImport->Update();
_byteSwaper->Swap8BERange(_image.data(), _image.size()/8);
qDebug() << _imageImport->GetOutput()->GetScalarRange()[1]; //Is not updated, always keeps the value of the first image
_bModeVolumeNode->SetSpacing(_pixelDepth, _pixelWidth, 1);
_bModeVolumeNode->Modified();
The display of the images works fine, I have a real time updated US image displayed on Slicer. But when I select the Volumes module Slicer crashes. I think it has something to do with the fact that the vtkImageData sent by the _imageImport is not updated, indeed _imageImport->GetOutput()->GetScalarRange()[1] always return the same value.
Any idea on how I could update this vtkImageData ?
Many thanks in advance.
Receiving real-time images from ultrasound systems is implemented using OpenIGTLink protocol. Real-time display is available with SlicerIGT (www.slicerigt.org - see tutorials).
Plus toolkit (www.plustoolkit.org) implements interface to a number of ultrasound systems (Ultrasonix, BK, Interson, Philips, Telemed, etc). If your device is not yet supported then we can help in getting it integrated.
Don’t spend time reinventing the wheel by implementing a new TCP/IP based protocol instead of OpenIGTLink, or reimplement OpenIGTLink and VolumeResliceDriver for real-time receiving and display.
Thanks for your answer, I am using a Verasonics scanner which, unless I am mistaken, is not supported by Plus. Furthermore Verasonics system uses Matlab for creating ultrasound sequences and I have not seen a full implementation of the OpenIGTLink protocole in Matlab.
I had a look at this : https://app.assembla.com/spaces/plus/wiki/Matlab_interface and I tried to implement a igtlSendImage function based on the existing (and working) igtlSendTransform function. Unfortunately my function causes Slicer to crash without any error messages.
I am currently thinking about creating a simple OpenIGTLink server in C++ with the OpenIGTLink library to debug my Matlab function, hoping I would get more indications about what is wrong in the message format I send.
Tell me if you see cleverer things to do. Many thanks !
If you want to keep using Matlab then you can use this Matlab OpenIGTLink implementation. Sending of images is slow, which could be addressed as described here. It would be great if you could fix it.
We would also welcome addition of Verasonics support to Plus. Let us know if you need help with that.
I have a transducer for ultrasound image called “Lumify” from Philps
It is working on the mobile device using the custom App from Philps
Dose this particular type of transducer also can directly link with 3D Slicer ?
If it is not is there any efficient way to connect this device to 3D Slicer ?
If you want to make Philips Lumify work then you need to first contact Philips if it is possible at all. The chances are very low, because most point-of-care ultrasound systems don’t provide real-time streaming interface. In case it is possible, then you may need to purchase it (they may ask a few $10k, mostly to cover cost of Philips engineering support) and you may need to set up a research contract (you may need to tell what you want to use the device for, etc. and it may be a complicated process that may take several months). If you get the API documentation then you need to implement the receiving side (get images, image metadata, and optionally send control parameters, such as depth, dynamic range, etc.), which may take several weeks for an experienced software developer and several months for a newcomer, costing about $10-15k overall.
Most likely it is much cheaper and faster to get a device that is already supported by Plus. For example the Clarius wireless ultrasound is quite similar to the Philips Lumify and it is supported by Plus. I don’t think they require a research contract, so there should not be much delay in getting access, and the cost of the real-time connection is about $10-15k (in addition to the purchase price of the device).
If a laptop-based system is acceptable then you can get a Telemed micrUS ext for under $5k (including real-time connection) and it is already supported by Plus.