Hello!.
I’m using a video reader to collect the live time images from an ultrasound machine.
I managed to collect that video data and stream it to a rtmp URL.
Now I guess I have to connect this URL with PlusToolkit, right? Or PlusToolkit is already able to detect the images captured by the video reader?
My idea is to send the frames from the streaming to SlicerIGT and then show those images in 3DSlicer.
Thanks in advance!
Try the OpenCV video device in Plus, it may be able to receive an rtmp stream. You can then follow SlicerIGT tutorials to set up receiving and visualization in Slicer.
Hi Andras! Thanks for your reply.
When I use that configuration file, it says that OpenCVVideo is not a supported Device.
|ERROR|008.246000|SERVER> Unknown device type: OpenCVVideo. Supported devices: 3dConnexion, AuroraTracker, ChRobotics, Epiphan, FakeTracker, GenericSerialDevice, ICCapturing, ImageProcessor, Microchip, MmfVideo, NDITracker, NoiseVideo, OpenIGTLinkTracker, OpenIGTLinkVideo, OpticalMarkerTracker, PhidgetSpatial, PolarisTracker, SavedDataSource, UsSimulator, VFWVideo, VirtualBufferedCapture, VirtualCapture, VirtualDiscCapture, VirtualMixer, VirtualSwitcher, VirtualVolumeReconstructor| in :\D\PSNP64b\PlusLib\src\PlusDataCollection\vtkPlusDeviceFactory.cxx(386)
|ERROR|008.264000|SERVER> No devices created. Please verify configuration file and any error produced.| in :\D\PSNP64b\PlusLib\src\PlusDataCollection\vtkPlusDataCollector.cxx(134)
|INFO|008.264000| Server process terminated.| in E:\D\PSNP64b\PlusApp\PlusServerLauncher\PlusServerLauncherMainWindow.cxx(453)
|INFO|008.265000| Disconnect request successful| in E:\D\PSNP64b\PlusApp\PlusServerLauncher\PlusServerLauncherMainWindow.cxx(307)
|ERROR|008.265000| Failed to start server process| in E:\D\PSNP64b\PlusApp\PlusServerLauncher\PlusServerLauncherMainWindow.cxx(219)
I’m using the last version of Plus Toolkit.
It seems I have to do a proper installation of Plus Toolkit, https://github.com/PlusToolkit/PlusLib/issues/363
I just downloaded a part of it. I’m following this instructions now https://github.com/PlusToolkit/PlusBuild/blob/master/Docs/BuildInstructionsWindows.md
I’m going to enable OpenCV video device in the Plus nightly build and see how much it increases the package size.
If it is not too much, then we can enable it in the downloadable releases.
That would be amazing! Yesterday I had a few issues trying to set up the PlusBuild (long build process, some errors after build, etc…)
Tell me if you do this update ^^
OpenCV was already included in package for ArUco marker tracking, so the additional size of the OpenCVVideo device was negligible.
The device should be included in the nightly packages starting tomorrow.
Hi again. Thanks @Sunderlandkyl for updating the nighty package.
I still have problems with the configuration in 3d Slicer. I’ll try to give all the possible information so maybe someone can help me out.
I’m currently streaming the ultrasound image in rtmp://192.168.146.30/live/1234. This works because I tried this URL in VLC and I can see the broadcast.
Now, I prepared this xml file for the plus server.
<PlusConfiguration version="2.1">
<DataCollection StartupDelaySec="1.0" >
<DeviceSet
Name="PlusServer: OpenCV device capturing an RTMP stream"
Description="Broadcasting acquired video through OpenIGTLink"
/>
<Device
Id="VideoDevice"
Type="OpenCVVideo"
RequestedCaptureApi="CAP_FFMPEG"
VideoURL="rtmp://192.168.146.30/live/1234">
<DataSources>
<DataSource Type="Video" Id="Video" ImageType="RGB_COLOR" PortUsImageOrientation="MF" />
</DataSources>
<OutputChannels>
<OutputChannel Id="VideoStream" VideoDataSourceId="Video" />
</OutputChannels>
</Device>
</DataCollection>
<CoordinateDefinitions>
<Transform From="Image" To="Reference"
Matrix="
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1" />
</CoordinateDefinitions>
<PlusOpenIGTLinkServer
MaxNumberOfIgtlMessagesToSend="1"
MaxTimeSpentWithProcessingMs="50"
ListeningPort="18944"
SendValidTransformsOnly="true"
OutputChannelId="VideoStream" >
<DefaultClientInfo>
<MessageTypes>
<Message Type="IMAGE" />
</MessageTypes>
<ImageNames>
<Image Name="Image" EmbeddedTransformToFrame="Reference" />
</ImageNames>
</DefaultClientInfo>
</PlusOpenIGTLinkServer>
</PlusConfiguration>
I did a little change to the .xml @lassoan suggested, I had to change the parameter StreamURL to VideoURL, because I was getting errors and the PLUS Server logs suggested me that.
When I Launch the server, I get 0 errors. However, I don’t know if Plus Server is reading correctly the images yet (In the logs I have several python debug lines, I’ll trace down them now, but maybe this is not the issue)
032519_104702.717|TRACE|045.826000| vtkPlusOpenCVCaptureVideoSource::InternalUpdate| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpenCVVideo\vtkPlusOpenCVCaptureVideoSource.cxx(275)
032519_104702.750|TRACE|045.860000| vtkPlusOpenCVCaptureVideoSource::InternalUpdate| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpenCVVideo\vtkPlusOpenCVCaptureVideoSource.cxx(275)
032519_104702.785|TRACE|045.894000| vtkPlusOpenCVCaptureVideoSource::InternalUpdate| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpenCVVideo\vtkPlusOpenCVCaptureVideoSource.cxx(275)
032519_104702.818|TRACE|045.927000| vtkPlusOpenCVCaptureVideoSource::InternalUpdate| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpenCVVideo\vtkPlusOpenCVCaptureVideoSource.cxx(275)
032519_104702.829|INFO|045.939000| Received new client connection (client 1 at 192.168.146.30:18944). Number of connected clients: 1| in E:\D\PTNP64b\PlusLib\src\PlusServer\vtkPlusOpenIGTLinkServer.cxx(277)
032519_104702.851|TRACE|045.960000| vtkPlusOpenCVCaptureVideoSource::InternalUpdate| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpenCVVideo\vtkPlusOpenCVCaptureVideoSource.cxx(275)
032519_104702.884|TRACE|045.994000| vtkPlusOpenCVCaptureVideoSource::InternalUpdate| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpenCVVideo\vtkPlusOpenCVCaptureVideoSource.cxx(275)
032519_104702.919|TRACE|046.028000| vtkPlusOpenCVCaptureVideoSource::InternalUpdate| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpenCVVideo\vtkPlusOpenCVCaptureVideoSource.cxx(275)
032519_104702.936|INFO|046.045000| OpenIGTLink broadcasting started. No data was available between 0-41.0331sec, therefore no data were broadcasted during this time period.| in E:\D\PTNP64b\PlusLib\src\PlusServer\vtkPlusOpenIGTLinkServer.cxx(428)
032519_104702.936|TRACE|046.045000| vtkPlusDevice::GetTrackedFrameList(41.1331, 1)| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(717)
032519_104702.936|TRACE|046.045000| Unable to get latest timestamp from video buffer!| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(1273)
032519_104702.936|TRACE|046.045000| Unable to get most recent timestamp!| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(754)
032519_104702.936|TRACE|046.045000| Number of added frames: 1 out of 1| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(935)
032519_104702.946|TRACE|046.055000| Failed to get tracked frame list from data collector (last recorded timestamp: 46.042243| in E:\D\PTNP64b\PlusLib\src\PlusServer\vtkPlusOpenIGTLinkServer.cxx(433)
032519_104702.953|TRACE|046.062000| vtkPlusOpenCVCaptureVideoSource::InternalUpdate| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\OpenCVVideo\vtkPlusOpenCVCaptureVideoSource.cxx(275)
032519_104702.983|TRACE|046.092000| vtkPlusDevice::GetTrackedFrameList(46.0422, 1)| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(717)
032519_104702.983|TRACE|046.092000| Unable to get latest timestamp from video buffer!| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(1273)
032519_104702.983|TRACE|046.092000| Unable to get most recent timestamp!| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(754)
032519_104702.983|TRACE|046.093000| Number of added frames: 1 out of 1| in E:\D\PTNP64b\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(935)
That is just a part of the logs when I connect the client from the 3D slicer. In Slicer, first I’m setting the IGTLink, following the tutorial steps:
After that, I press the box “Active”. Then I go to the Plus Remote Module. Here is where I start getting lost because Slicer is asking me to set a Configuration file to “Launch Server”? Why would I Launch another Server when the plus one is already running? 3DSlicer should play a client role, right? I guess I have a misconception about this…
Anyways I set the IP and the port, and for the configuration file, I’m using the Plus Server one. Then I click connect and I open the log, but when I click “Launch server”, nothings happens.
Do you guys know what I might be missing here? D: Also, I saw the Capture device ID field, but I’m not able to write in it.
Once you click “Active” then your images should already start appearing in 3D Slicer as a volume called “Image_Reference”.
You don’t need to use the “Plus server launcher remote control” section in PlusRemote. It is an interface that can be used to start and stop PlusServer instances remotely.
Hi @Sunderlandkyl you were right, I don’t need Plus Remote method for this. Sorry.
However, I’m not able to see the images in real time properly. I managed to take some snapshots by using the method “UltrasoundSnapshots”. When I click “Add Snapshot”, I can see the last frame of my stream.
But I don’t find the way to see the whole streaming.
If I enter the method Wizards>“Compare Volumes” and I press “Lightbox Target Volume” I can see the streaming, but in 9 boxes and exiting the working space…
I guess the Image_Reference is a vector of 1 frame and each time I collect a new frame, I’m updating it?
Ok, it was as easy as Data > Image_Reference > Show , then link the different views and toggle 3D view.
Finally!
I’m glad you could figure it out. You can find step-by-step instructions for setting up Slicer scenes for image guidance in SlicerIGT tutorials.