How can I record multiple video inputs at the same time from Slicer?

lb123 · December 6, 2021, 2:36pm

Hi,

I’m broadcasting 2 acquired videos through OpenIGTLink from Plus Server to Slicer 4.13.0.
The videos consist in live data coming from 2 epiphan frame grabbers.

I can correctly see both the live videos on Slicer.

I would like to record both them from Slicer.

I tried to record adding a CaptureDevice in the plus server configuration file:

<Device
  Id="CaptureDevice"
  Type="VirtualCapture"
  BaseFilename="RecordingTest.igs.mha"
  EnableCapturingOnStart="FALSE" >
  <InputChannels>
    <InputChannel Id="VideoStream1" />
<InputChannel Id="VideoStream2" />
  </InputChannels>
</Device>

Then I tried to start a recording from Slicer using the recording button of Plus Remote (IGT module) but only the VideoStream1 is recorded.

Can you please tell me what I am doing wrong?

Sunderlandkyl · December 6, 2021, 4:06pm

Each VirtualCapture device can only record a single video stream, you’ll need to have one for each.

lb123 · December 6, 2021, 4:26pm

Thank you for your reply. So I should add a second VirtualCaputure device like shown below, right?

<Device
  Id="CaptureDevice1"
  Type="VirtualCapture"
  BaseFilename="RecordingTest1.igs.mha"
  EnableCapturingOnStart="FALSE" >
  <InputChannels>
    <InputChannel Id="VideoStream1" />
  </InputChannels>
</Device>

<Device
  Id="CaptureDevice2"
  Type="VirtualCapture"
  BaseFilename="RecordingTest2.igs.mha"
  EnableCapturingOnStart="FALSE" >
  <InputChannels>
    <InputChannel Id="VideoStream2" />
  </InputChannels>
</Device>

But then how is it possible to run 2 recordings at the same time?

As you can see from the attached image it is possibile to select just one capture device at a time

Is it possible to do it just coding or is there the possibility to do it from the Slicer GUI?

Maybe the most immediate way is just to set EnableCapturingOnStart=“TRUE”, right?

Also, is it possible to reduce the size of .mha files? Huge files are generated (several GB for a few seconds of recording).

Sunderlandkyl · December 6, 2021, 5:53pm

There doesn’t seem to be a way to do this from the GUI currently.

EnableCapturingOnStart=“TRUE” will work if you’re ok with recording everything once the server is started.

You can also start recording from python like this:

Start:

startCommand = slicer.vtkSlicerOpenIGTLinkCommand()
startCommand.SetName("StartRecording")
startCommand.SetCommandContent('<Command Name="StartRecording" CaptureDeviceId="CaptureDevice" OutputFilename="Recording.igs.mha" EnableCompression="False"></Command>')
igtlConnectorNode.SendCommand(startCommand)

Stop

stopCommand = slicer.vtkSlicerOpenIGTLinkCommand()
stopCommand.SetName("StopRecording")
stopCommand.SetCommandContent('<Command Name="StopRecording" CaptureDeviceId="CaptureDevice"></Command>')
igtlConnectorNode.SendCommand(stopCommand)

lb123 · December 6, 2021, 5:57pm

Thank you a lot for your explanation!

Do you know if there is any way to decrease the size of the recordings?
Huge files are generated (several GB for a few seconds of recording).

Sunderlandkyl · December 6, 2021, 7:40pm

Couple of ways:

Can you post the epiphan device config? If the image is in color, then it will increase the size of the files.
You can add "EnableCompression=“TRUE” to the command to use compression when saving.
In tomorrow’s nightly Plus build, you can change the file name extension from “.igs.mha” to “mkv”. Even if the video is color, this should reduce file size by saving as a Matroska file (using VP9 compression).

Not sure if you are aware, but you can also record images and transforms in Slicer using the Sequences module. Let me know if you have any questions.

lb123 · December 16, 2021, 4:32pm

Thank you, I tried to save to “mkv” and the size is really smaller now!

So, following your suggestions I recorded 2 videos in mkv format. But now I would like to replay them but I am facing some issues.

This is the configuration file I am using to replay the 2 videos:

 <PlusConfiguration version="2.1">

  <DataCollection StartupDelaySec="1.0" >
    <DeviceSet
      Name="test_multi_replay_mkv___"
      Description="Broadcasting acquired video through OpenIGTLink"
    />
	<Device
		Id="VideoDevice"
		Type="SavedDataSource"
		SequenceFile="RecordingFristVideo21-12-16-10-20-46.mkv"
		UseData="IMAGE"
		UseOriginalTimestamps="TRUE"
		ToolReferenceFrame="Reference"
		RepeatEnabled="TRUE" >
		<DataSources>
		<DataSource Type="Video" Id="Video" BufferSize="100" />
		</DataSources>
		<OutputChannels>
		<OutputChannel Id="VideoStream" VideoDataSourceId="Video" />
		</OutputChannels>
    </Device>
	
	<Device
		Id="VideoDevice2"
		Type="SavedDataSource"
		SequenceFile="RecordingSecondVideo21-12-16-10-20-46.mkv"
		UseData="IMAGE"
		UseOriginalTimestamps="TRUE"
		ToolReferenceFrame="Reference"
		RepeatEnabled="TRUE" >
		<DataSources>
		<DataSource Type="Video" Id="Video2" BufferSize="100" />
		</DataSources>
		<OutputChannels>
		<OutputChannel Id="VideoStream2" VideoDataSourceId="Video2" />
		</OutputChannels>
    </Device>
	
  </DataCollection>

  <CoordinateDefinitions>
    <Transform From="Image" To="Reference"
      Matrix="
        1 0.0 0.0 0.0
        0.0 1 0.0 0.0
        0.0 0.0 1 0.0
        0 0 0 1" />
		
	<Transform From="Image2" To="Reference"
      Matrix="
        1 0.0 0.0 0.0
        0.0 1 0.0 0.0
        0.0 0.0 1 0.0
        0 0 0 1" />
  </CoordinateDefinitions>

  <PlusOpenIGTLinkServer
    MaxNumberOfIgtlMessagesToSend="1"
    MaxTimeSpentWithProcessingMs="50"
    ListeningPort="18944"
    SendValidTransformsOnly="true"
    OutputChannelId="VideoStream" >
    <DefaultClientInfo>
      <MessageTypes>
        <Message Type="IMAGE" />
      </MessageTypes>
      <ImageNames>
        <Image Name="Image" EmbeddedTransformToFrame="Reference" />
      </ImageNames>
    </DefaultClientInfo>
  </PlusOpenIGTLinkServer>
  
    <PlusOpenIGTLinkServer
    MaxNumberOfIgtlMessagesToSend="1"
    MaxTimeSpentWithProcessingMs="50"
    ListeningPort="18945"
    SendValidTransformsOnly="true"
    OutputChannelId="VideoStream2" >
    <DefaultClientInfo>
      <MessageTypes>
        <Message Type="IMAGE" />
      </MessageTypes>
      <ImageNames>
        <Image Name="Image2" EmbeddedTransformToFrame="Reference" />
      </ImageNames>
    </DefaultClientInfo>
  </PlusOpenIGTLinkServer>

</PlusConfiguration>

When I run the OpenIGTLink clients in Slicer as you can see in the screenshot below there isn’t any vector volume on the “IGTLConnector” but only on the “IGTLConnector_1”:

Do you know I to solve this issue?

Thank you

J.vd.Zee · March 19, 2024, 8:05am

Hi!
Can I re-open this topic?
I would like to make multiple streams of the same videodevice, but each with a different crop. I’m now struggling with cropping each individual stream.
Expected behaviour:
Cropping each videostream with the specific crop.

Observerd behavior
Cropping each videostream with the first cropping settings.

Can someone help me out?

Sunderlandkyl · March 19, 2024, 3:00pm

I’m not sure if the MmfVideo device will work with multiple data sources, but a couple of things:

You should only include one “DataSources” element. Both “DataSource” should be in the same one.
Same deal with “OutputChannels” and “OutputChannel”.
A channel can only include a single image stream, so you would need to separate the two. (VideoStream1, VideoStream2). Probably using more descriptive names for each stream.

J.vd.Zee · March 19, 2024, 3:14pm

Hi Kyle,

Thank you for your prompt reply!

I tried your adjustments (see the photo). However, it is not working. Do you have other possible adjustments?

Sunderlandkyl · March 19, 2024, 3:17pm

You will need to provide more information. It what way is it not working?

J.vd.Zee · March 20, 2024, 3:17pm

In my previous post, I included the tips you mentioned.
However, the Plus gives an error after launching:

|WARNING|61724.344000|SERVER> vtkPlusMmfVideoSource is expecting one output channel and there are 2 channels. First output channel will be used.

Is it possible to create different VideoDevices for each videostream (same capture device) and specific crop?

Sunderlandkyl · March 20, 2024, 4:38pm

Ok, thanks! Since it looks the device doesn’t currently support multiple output channels, then it is probably not currently possible. I’ll look into what is required to support it.

Sunderlandkyl · March 20, 2024, 4:54pm

I’ve updated the vtkPlusMmfVideoSource device. The nightly release tomorrow should be able to support multiple video data sources at the same time, with different crop regions.

J.vd.Zee · March 22, 2024, 10:11am

Hi Kyle!

Thanks for updating.
However, the new update is not working yet. I added my configuration file in the attachments. I made two VideoDevices for each videostream, same capture device and specific crop. Can you help me out?

Sunderlandkyl · March 22, 2024, 1:35pm

The config file should have only 1 Device that has 2 Data sources and 2 Output channels.

J.vd.Zee · March 25, 2024, 11:59am

It worked!
Thanks for helping me out @Sunderlandkyl

Topic		Replies	Views
How to connect two video streams into slicer using plus server Support plus	8	111	April 22, 2024
Slicer crashes when I try to record 2 videos at the same time Support plus , slicerigt , openigtlink	1	364	December 22, 2021
Unable to input channel Id in Slicer->IGT->Plus Remote Support surgical-navigation	14	1006	March 8, 2019
Slicer IGT Multiple Tools, Plus ToolKit Support slicerigt , openigtlink	1	414	June 24, 2021
Replay recorded data as a sequence in the SequenceBrowser Support surgical-navigation , sequences	1	627	December 21, 2021

How can I record multiple video inputs at the same time from Slicer?

Related topics