Operating system: Windows 10
Slicer version: 4.11.20200930
Plus version: 2.9.0.20240320-Telemed-Win64
Expected behavior: Image slice from Telemed moves as tracked by the NDI Aurora. Scout scan projects movement and live reconstruction creates volume.
Actual behavior: Image slice from Telemed is not moving based on NDI Aurora tracker. Scout scan is not working, and live reconstruction does not work due to scout scan.
Hi, I am trying to perform live reconstruction of the kidney using Telemed to visualize images and NDI Aurora to track movement of probe. I successfully connected the devices to the Plus server using the configuration file provided below. I combined each device’s respective configuration into one file and followed the steps in this video. However, each time I get to the Scout scan and live reconstruction part, it does not work. Based on documentation, tutorials, and videos, the image slice projected by Telemed should move based on the movement of the probe tracked by the NDI Aurora. From my impression, the scout scanning and live reconstruction occurs based on the movement of the probe. The video uses a different method than the SlicerIGT tutorials. The video uses the Plus Remote module to set up the server and rearranges the transform data hierarchy. I assume that I need the Image_Image transform in my hierarchy but my list only contains Image_Reference. Also, when I click on the view in the 3D volume section, the image slice takes up the whole outlined volume box and the volume box disappears. I have included some screenshots as reference. I have trying to achieve the same results as this video, but with the Telemed and NDI Aurora. I looked at forums to find similar questions and tried to troubleshoot it, but I cannot figure it out. I’m not sure if my configuration files are incorrect or missing a part or if I am missing a step when setting up the live reconstruction in 3D Slicer. Please provide advice and solutions to this problem.
Please don’t upload images of config files. It makes them difficult to search.
Your ImageToReference is a static transform, so the ultrasound image will never move. You’ll need to update your config file to use the correct names for the tools tracked by the Aurora.
With only one sensor you can perform the volume reconstruction in the “Tracker” coordinate system, provided that the object you’re scanning never moves, but it would be better to have both a reference sensor attached to the object and a sensor on the probe, and perform the reconstruction in the “Reference” coordinate system.
If you want to perform the volume reconstruction like it is done in the first video, you will need the latest stable or nightly version of Slicer.
Hi Kyle,
I appreciate your timely response. Sorry about the configuration file image. I didn’t know how to upload it since I could only upload image or video files and thought pasting it into the text would make the post too long. Is there a better way to share the file for next time?
Thank you for the resource! The sketch is very helpful and clarifies the problem. Is there a configuration file based on this coordinate system sketch?
I downloaded the latest stable version of Slicer and modified the given config file for the NDI Aurora and Telemed but I still cannot get the slice to move. I included the modified config file and an image of my setup (currently testing on gel sample instead of phantom model). Also, initially, I did see ProbetoTracker and ReferencetoTracker in my hierarchy list but the only thing listed is the Image_Reference. I’m assuming it has something to do with the transform matrix in the coordinate definitions section. Should I have used the matrix given in the default Telemed config file?
Config file
<PlusConfiguration version="2.1">
<DataCollection StartupDelaySec="1.0" >
<DeviceSet
Name="PlusServer: Telemed US + NDI Aurora tracker (Probe, Reference)"
Description="Broadcasting ultrasound images acquired from the Ultrasonix system through OpenIGTLink. Probe is spatially calibrated for any imaging depth. If PlusServer does not run on the Ultrasonix PC then update the IP attribute in the Device element with the Ultrasonix PC's IP address. Ascension3DG sensors should be plugged in to the Ascension3DG DriveBay mounted on Ultrasonix US in the following order from to leftmost slot (Transducer 1) to the right: 1 Probe, 2 Reference, 3 Stylus." />
<Device
Id="TrackerDevice"
Type="AuroraTracker"
SerialPort="3"
BaudRate="9600"
AcquisitionRate="50"
LocalTimeOffsetSec="0.0"
FilterAcWideNotch="1"
ToolReferenceFrame="Tracker" >
<DataSources>
<DataSource
Type="Tool"
Id="Probe"
RomFile="DDRO-080-061-01_GENERIC (4).rom"
PortName="0"
/> <!-- PortName relates to physical port 1 -->
<DataSource
Type="Tool"
Id="Reference"
RomFile="DDRO-080-051-01_GENERIC.rom"
PortName="1"
/> <!-- PortName relates to physical port 2 -->
</DataSources>
<OutputChannels>
<OutputChannel Id="TrackerStream" >
<DataSource Id="Probe"/>
<DataSource Id="Reference"/>
</OutputChannel>
</OutputChannels>
</Device>
<Device
Id="VideoDevice"
Type="TelemedVideo"
AcquisitionRate="30"
LocalTimeOffsetSec="-0.2976"
IP="127.0.0.1"
AutoClipEnabled="TRUE"
ImageToTransducerTransformName="ImageToTransducer" >
<DataSources>
<DataSource Type="Video" Id="Video" PortUsImageOrientation="UN" />
</DataSources>
<OutputChannels>
<OutputChannel Id="VideoStream" VideoDataSourceId="Video"/>
</OutputChannels>
</Device>
<Device
Id="TrackedVideoDevice"
Type="VirtualMixer" >
<InputChannels>
<InputChannel Id="TrackerStream" />
<InputChannel Id="VideoStream" />
</InputChannels>
<OutputChannels>
<OutputChannel Id="TrackedVideoStream"/>
</OutputChannels>
</Device>
<Device
Id="CaptureDevice"
Type="VirtualCapture"
BaseFilename="Recording.igs.mhd"
EnableCapturingOnStart="FALSE"
RequestedFrameRate="15" >
<InputChannels>
<InputChannel Id="TrackedVideoStream" />
</InputChannels>
</Device>
<Device
Id="VolumeReconstructorDevice"
Type="VirtualVolumeReconstructor"
OutputVolDeviceName="Volume_Reference"
EnableReconstruction="FALSE" >
<InputChannels>
<InputChannel Id="TrackedVideoStream" />
</InputChannels>
<VolumeReconstruction
ImageCoordinateFrame="Image" ReferenceCoordinateFrame="Reference"
CompoundingMode="MEAN" Interpolation="LINEAR"
Optimization="FULL" NumberOfThreads="2"
ClipRectangleOrigin="0 0" ClipRectangleSize="820 616" PixelRejectionThreshold="1"
OutputSpacing="0.5 0.5 0.5"
FillHoles="ON" >
<HoleFilling>
<HoleFillingElement Type="GAUSSIAN" Size="5" Stdev="0.6667" MinimumKnownVoxelsRatio="0.50001" />
<HoleFillingElement Type="STICK" StickLengthLimit="9" NumberOfSticksToUse="1" />
</HoleFilling>
</VolumeReconstruction>
</Device>
</DataCollection>
<CoordinateDefinitions>
<!-- transform matrix provided by Telemed default config file -->
<Transform From="Image" To="Reference"
Matrix="
0.2 0.0 0.0 0.0
0.0 0.2 0.0 0.0
0.0 0.0 0.2 0.0
0 0 0 1" />
</CoordinateDefinitions>
<PlusOpenIGTLinkServer
MaxNumberOfIgtlMessagesToSend="1"
MaxTimeSpentWithProcessingMs="50"
ListeningPort="18944"
SendValidTransformsOnly="true"
OutputChannelId="TrackedVideoStream" >
<DefaultClientInfo>
<MessageTypes>
<Message Type="IMAGE" />
<Message Type="TRANSFORM" />
</MessageTypes>
<TransformNames>
<Transform Name="ProbeToReference" />
<Transform Name="ReferenceToTracker" />
</TransformNames>
<ImageNames>
<Image Name="Image" EmbeddedTransformToFrame="Reference" />
</ImageNames>
</DefaultClientInfo>
</PlusOpenIGTLinkServer>
</PlusConfiguration>
Thank you for your advice. Unfortunately, my problem still remains with the Transform hierarchy list. I do not see the other transforms. The only one that is being listed is Image_Probe. How do I get the other transforms to appear in the list like in the image you show?
Update: I have ReferenceToTracker in my list by opening OpenIGTLinkIF first and then viewing Data; previously, I was following the steps of the video and creating the Plus Remote Server and then viewing Data. However, I am missing Image_Reference and ProbeToReference in my list. Any way to solve this issue?
I am still having issues with reconstructing the volume in 3D Slicer. The image slice is now moving based on the position of the transducer. However, after I complete the Scout scan, there is a black square. I ended up following the steps in the SlicerIGT tutorial presentation instead of following the video in my original post (it wasn’t working as well and produced nothing). How do I fix this and achieve the desired reconstruction of the kidney? Any suggestions on how to visualize the reconstruction volume?
