Hello!
I am trying to connect a GE ultrasound machine to the Plus framework for free-hand calibration (fCal). I am using Slicer to check that the data is valid. I wrote a small program in C++ that sends image data using OpenIGTLink. The image data is originally captured using GE AppAPI interface. Using this program, I am successfully able to stream directly to Slicer using the OpenIGTLinkIF module:
bool StreamLink::sendImage(const cv::Mat& frame, uint64_t timestamp_ms)
{
if (socket_.IsNull())
return false;
cv::Mat img = frame.clone();
// Convert BGR -> RGB
int channels = img.channels();
if (channels == 3)
{
cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
}
//----------------------------------
// IMAGE SETUP
//----------------------------------
int size[3] = { img.cols, img.rows, 1 };
auto imageMsg = igtl::ImageMessage::New();
imageMsg->SetHeaderVersion(IGTL_HEADER_VERSION_1);
imageMsg->SetDimensions(size);
int svsize[3] = { img.cols, img.rows, 1 };
int svoffset[3] = { 0, 0, 0 };
imageMsg->SetSubVolume(svsize, svoffset);
float spacing[3] = { 1.0f, 1.0f, 1.0f };
imageMsg->SetSpacing(spacing);
imageMsg->SetNumComponents(channels);
imageMsg->SetScalarType(igtl::ImageMessage::TYPE_UINT8);
igtl::Matrix4x4 matrix;
igtl::IdentityMatrix(matrix);
matrix[1][1] = -1;
matrix[1][3] = img.rows - 1;
imageMsg->SetMatrix(matrix);
//----------------------------------
// Coordinate system
//----------------------------------
imageMsg->SetCoordinateSystem(igtl::ImageMessage::COORDINATE_LPS);
//----------------------------------
// Device name
//----------------------------------
imageMsg->SetDeviceName("Video");
//----------------------------------
// Allocate image buffer
//----------------------------------
imageMsg->AllocateScalars();
memcpy(
imageMsg->GetScalarPointer(),
img.data,
img.total() * img.elemSize()
);
//----------------------------------
// Timestamp
//----------------------------------
igtl::TimeStamp::Pointer ts = igtl::TimeStamp::New();
uint64_t sec = timestamp_ms / 1000;
uint64_t nsec = (timestamp_ms % 1000) * 1000000;
ts->SetTime(sec, nsec);
imageMsg->SetTimeStamp(ts);
static uint32_t frameNumber = 0;
frameNumber++;
imageMsg->SetMetaDataElement("FrameNumber", frameNumber);
//----------------------------------
// Pack + send
//----------------------------------
imageMsg->Pack();
int sent = socket_->Send(
imageMsg->GetPackPointer(),
imageMsg->GetPackSize()
);
//std::cout << "Sent frame "
// << img.cols << "x" << img.rows
// << " channels=" << channels
// << " bytes=" << imageMsg->GetPackSize()
// << std::endl;
return sent > 0;
}
However, when integrating it with the Plus framework I can connect to my OpenIGTLink server, but I am no longer able to receive the data in Slicer. I am using this config file in the PlusToolkit:
<PlusConfiguration version="2.9">
<DataCollection StartupDelaySec="1.0">
<DeviceSet
Name="fCal: GE ultrasound + Atracsys"
Description="Broadcasting ultrasound data from GE HealthCare + tracking data from Atracsys spryTrack180, fusionTrack250 or fusionTrack500 through OpenIGTLink."
/>
<Device
Id="TrackerDevice"
Type="AtracsysTracker"
MaxMissingFiducials="1"
MaxMeanRegistrationErrorMm="1.0"
ToolReferenceFrame="Tracker" >
<DataSources>
<DataSource Type="Tool" Id="Stylus" TrackingType="PASSIVE" GeometryFile="AtracsysTools/geometry420.ini" GeometryId="420" />
<DataSource Type="Tool" Id="Probe" TrackingType="PASSIVE" GeometryFile="AtracsysTools/geometry505.ini" GeometryId="505" />
</DataSources>
<OutputChannels>
<OutputChannel Id="TrackerStream">
<DataSource Type="Tool" Id="Stylus" />
<DataSource Type="Tool" Id="Probe" />
</OutputChannel>
</OutputChannels>
</Device>
<!-- ========================= -->
<!-- ULTRASOUND (FROM C++ IGTL SERVER) -->
<!-- ========================= -->
<Device
Id="VideoDevice"
Type="OpenIGTLinkVideo"
MessageType="IMAGE"
ServerAddress="127.0.0.1"
ServerPort="18945"
Active="true"
IgtlMessageCrcCheckEnabled="false"
LocalTimeOffsetSec="0"
ReceiveTimeoutSec="5"
ReconnectOnReceiveTimeout="true">
<DataSources>
<DataSource Id="Video" Type="Video" PortUsImageOrientation="MF" />
</DataSources>
<OutputChannels>
<OutputChannel Id="VideoStream"
VideoDataSourceId="Video"/>
</OutputChannels>
</Device>
<!-- ========================= -->
<!-- FUSION (TRACKED VIDEO) -->
<!-- ========================= -->
<Device
Id="TrackedVideoDevice"
Type="VirtualMixer"
ToolReferenceFrame="Tracker" >
<InputChannels>
<InputChannel Id="TrackerStream"/>
<InputChannel Id="VideoStream"/>
</InputChannels>
<OutputChannels>
<OutputChannel Id="TrackedVideoStream"/>
</OutputChannels>
</Device>
</DataCollection>
<!-- ========================= -->
<!-- OPENIGTLINK SERVER (TO SLICER) -->
<!-- ========================= -->
<PlusOpenIGTLinkServer
ListeningPort="18944"
OutputChannelId="TrackedVideoStream"
SendValidTransformsOnly="TRUE">
<DefaultClientInfo>
<MessageTypes>
<Message Type="IMAGE"/>
<Message Type="TRANSFORM"/>
</MessageTypes>
</DefaultClientInfo>
<TransformNames>
<Transform Name="StylusToTracker"/>
<Transform Name="ProbeToTracker"/>
</TransformNames>
<ImageNames>
<Image Name="Video" EmbeddedTransformToFrame="Probe" />
</ImageNames>
</PlusOpenIGTLinkServer>
</PlusConfiguration>
And I receive these errors in the log, which indicate that maybe something is wrong with my timestamps, I just don’t know what:
052226_123756.913|INFO|011.913000| Received new client connection (client 1 at 127.0.0.1:18944). Number of connected clients: 1| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusServer\vtkPlusOpenIGTLinkServer.cxx(298)
052226_123756.927|TRACE|011.927000| vtkPlusAtracsysTracker::InternalUpdate| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusDataCollection\Atracsys\vtkPlusAtracsysTracker.cxx(716)
052226_123756.927|INFO|011.927000| OpenIGTLink broadcasting started. No data was available between 0-5.514sec, therefore no data were broadcasted during this time period.| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusServer\vtkPlusOpenIGTLinkServer.cxx(449)
052226_123756.927|TRACE|011.927000| vtkPlusDevice::GetTrackedFrameList(5.614, 50)| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(719)
052226_123756.927|TRACE|011.927000| Unable to get latest timestamp from video buffer!| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(1275)
052226_123756.927|TRACE|011.927000| Failed to get video buffer item UID from time: 11.911000| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(1427)
052226_123756.927|TRACE|011.927000| Failed to get video buffer timestamp from UID: 268| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(1429)
052226_123756.927|TRACE|011.927000| Unable to get most recent timestamp!| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(756)
052226_123756.927|TRACE|011.927000| Number of added frames: 50 out of 50| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusDataCollection\vtkPlusChannel.cxx(937)
052226_123756.934|TRACE|011.935000| Failed to get tracked frame list from data collector (last recorded timestamp: 11.863000| in D:\a\PlusLib\PlusLib\build\PlusLib\src\PlusServer\vtkPlusOpenIGTLinkServer.cxx(454)