Data node display nodes not maintained when creating a sequence

Support
1

I’ve been having some issues creating a sequence where the display nodes of the given data nodes that I’m adding are not being maintained. I have a set of loaded 2D images where I have customized each of their display volumes with their appropriate colormap, window/level min/max and lower/upper threshold. However when I select one of my volumes to add as a data node to my sequence node I observe that all my display customizations have been lost.

Am I using sequences incorrectly? Or is the expectation to set up the display for each data node after it has been added to the sequence?

Below is some sample code of loading 2 MRHead volumes and customizing one of them, but when selecting the SequenceNode to show in the slice viewers I observe that the display customizations I did at the beginning were lost. I also observe this when selecting the SequenceNode in the Volumes module and playing the sequence as I don’t see the lower threshold value changing.

# Load sample volume
import SampleData
sampleDataLogic = SampleData.SampleDataLogic()
mrHead = sampleDataLogic.downloadMRHead()
mrHead2 = sampleDataLogic.downloadMRHead()
mrHead2.GetDisplayNode().SetAndObserveColorNodeID("vtkMRMLColorTableNodeRed")
mrHead2.GetDisplayNode().ApplyThresholdOn()
mrHead2.GetDisplayNode().SetLowerThreshold(79)

sequence_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSequenceNode", "MySequenceNode")
sequence_node.SetDataNodeAtValue(mrHead, "0")
frame_vol_0 = sequence_node.GetNthDataNode(0)
frame_vol_0.GetDisplayNode()
sequence_node.SetDataNodeAtValue(mrHead2, "1")
frame_vol_1 = sequence_node.GetNthDataNode(1)
frame_vol_1.GetDisplayNode()

browser_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSequenceBrowserNode", "MyBrowserNode")
browser_node.AddSynchronizedSequenceNode(sequence_node)
2

I think what is probably happening here is that display nodes are not “attached” to the scalar volume node as it is added into the sequence. Instead, browsing through the sequence node only copies the image data stored in the sequence into the scalar volume proxy node associated with the sequence. That proxy node is then displayed with the display node associated with the proxy node rather than with the display node associated with the scalar volume at the point when it was added to the sequence. I think you can probably create a sequence of display nodes and set the scalar volume proxy node to observe the proxy display node associated with the sequence of display nodes you want to run through. That way, when you step through the sequence browser, both the scalar volume node and the display node associated with it would be updated with the next value in each sequence. I’ve never done this for display nodes, but this is the way it works for transform nodes. For example, if you have a sequence of transforms to apply to a sequence of images, both need to be synchronized under the same sequence browser node, and the image proxy volume needs to have as its parent transform the proxy node of the transform sequence.

1 Like
3

From reading about the Basic MRML node types, it seems like Sequence nodes only store a list of “Data nodes”. A “Data node” is a Volume/Model/Segmentation/Markups/Transform/Text/Table, however a display node is a separate category of MRML node.

I have a situation of a sequence of volume nodes where I have percentage based calculated Window/Levels and Threshold values for each of the volumes nodes. It seems like currently I would have to apply the same Window/Level and Threshold for each index in the sequence.

4

Another option is just to listen for events from the sequence browser node and update the volume display with your custom properties. This is basically what is done in the Animator.

5

Yes. It is up to you how you want to display the sequence. Probably a new display node is created by default but if you want you can copy the display node content of one of the source data nodes. You can also put the display node into a sequence - if you want display properties to change in time, too.

There is no such limitation. The beauty of the sequences infrastructure that it can work with any node type (not just data nodes but display nodes, transforms, views, etc.). The only requirement is that the class must implement copy content method.

Yes, it works well. You can select a display node as proxy node, start recording, modify display settings, stop recording, and then you can click play to see all your display changes.

6

@lassoan I guess I was confused by the term “data node” mentioned in the sequences documentation and how the term “data node” appears to exclude Display/Storage/View/Plot/SubjectHierarchy nodes as described in Basic MRML node types.

Below is my attempt at putting the display node into a sequence. However I’m observing that the sequence is not observing the change appropriately. I am doing this in code because I can’t seem to find a GUI method for setting up a display sequence.

image

# Load sample volume
import SampleData
sampleDataLogic = SampleData.SampleDataLogic()
mrHead = sampleDataLogic.downloadMRHead()
mrHead2 = sampleDataLogic.downloadMRHead()
mrHead2.GetDisplayNode().SetAndObserveColorNodeID("vtkMRMLColorTableNodeRed")
mrHead2.GetDisplayNode().ApplyThresholdOn()
mrHead2.GetDisplayNode().SetLowerThreshold(79)

sequence_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSequenceNode", "MySequenceNode")
sequence_node.SetDataNodeAtValue(mrHead, "0")
sequence_node.SetDataNodeAtValue(mrHead2, "1")

sequence_display_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSequenceNode", "MySequenceDisplayNode")
sequence_display_node.SetDataNodeAtValue(mrHead.GetDisplayNode(), "0")
sequence_display_node.SetDataNodeAtValue(mrHead2.GetDisplayNode(), "1")

browser_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSequenceBrowserNode", "MyBrowserNode")
browser_node.AddSynchronizedSequenceNode(sequence_node)
browser_node.AddSynchronizedSequenceNode(sequence_display_node)
volume_proxy_node = browser_node.GetProxyNode(sequence_node)
display_proxy_node = browser_node.GetProxyNode(sequence_display_node)
volume_proxy_node.SetAndObserveDisplayNodeID(display_proxy_node.GetID())

slicer.modules.sequences.toolBar().setActiveBrowserNode(browser_node)
7

I was wrong here. Things are working appropriately. I confused myself by not doing slicer.util.setSliceViewerLayers(background=volume_proxy_node) at the end of the code snippet I provided in my last comment so I wasn’t actually looking at the volume node I was thinking. I had set the browser node to be active, but I hadn’t shown it yet in the slice views.

8

Actually I’m wrong again. The colormap is updating appropriately for each index, but the Window/Level and Threshold properties are not appropriately changing in the sequence.

After the SetDataNodeAtValue call for a vtkMRMLScalarVolumeDisplayNode, I can confirm that Color node is copied over, but not window/level and not threshold. As seen by the output below produced by the included code in this comment.

Index 0 color node correct: True
Index 0 window/level min correct: False
Index 0 window/level max correct: False
Index 0 LowerThreshold correct: False
Index 0 UpperThreshold correct: False
Index 1 color node correct: True
Index 1 window/level min correct: False
Index 1 window/level max correct: False
Index 1 LowerThreshold correct: False
Index 1 UpperThreshold correct: False

# Load sample volume
import SampleData
sampleDataLogic = SampleData.SampleDataLogic()
mrHead = sampleDataLogic.downloadMRHead()
mrHead.GetDisplayNode().ApplyThresholdOn()
mrHead.GetDisplayNode().SetAutoWindowLevel(False)
mrHead.GetDisplayNode().SetWindowLevelMinMax(10, 120)
min, max = mrHead.GetImageData().GetScalarRange()
mrHead.GetDisplayNode().SetThreshold(0, max)

mrHead2 = sampleDataLogic.downloadMRHead()
mrHead2.GetDisplayNode().SetAndObserveColorNodeID("vtkMRMLColorTableNodeRed")
mrHead2.GetDisplayNode().ApplyThresholdOn()
min, max = mrHead2.GetImageData().GetScalarRange()
mrHead2.GetDisplayNode().SetThreshold(79, max)
mrHead2.GetDisplayNode().ApplyThresholdOn()
mrHead2.GetDisplayNode().SetAutoWindowLevel(False)
mrHead2.GetDisplayNode().SetWindowLevelMinMax(20, 100)

sequence_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSequenceNode", "MySequenceNode")
sequence_node.SetDataNodeAtValue(mrHead, "0")
sequence_node.SetDataNodeAtValue(mrHead2, "1")

sequence_display_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSequenceNode", "MySequenceDisplayNode")
sequence_display_node.SetDataNodeAtValue(mrHead.GetDisplayNode(), "0")
sequence_display_node.SetDataNodeAtValue(mrHead2.GetDisplayNode(), "1")
a = sequence_display_node.GetDataNodeAtValue("0")
print(f"Index 0 color node correct: {a.GetColorNodeID() == mrHead.GetDisplayNode().GetColorNodeID()}")
print(f"Index 0 window/level min correct: {a.GetWindowLevelMin() == mrHead.GetDisplayNode().GetWindowLevelMin()}")
print(f"Index 0 window/level max correct: {a.GetWindowLevelMax() == mrHead.GetDisplayNode().GetWindowLevelMax()}")
print(f"Index 0 LowerThreshold correct: {a.GetLowerThreshold() == mrHead.GetDisplayNode().GetLowerThreshold()}")
print(f"Index 0 UpperThreshold correct: {a.GetUpperThreshold() == mrHead.GetDisplayNode().GetUpperThreshold()}")
b = sequence_display_node.GetDataNodeAtValue("1")
print(f"Index 1 color node correct: {b.GetColorNodeID() == mrHead2.GetDisplayNode().GetColorNodeID()}")
print(f"Index 1 window/level min correct: {b.GetWindowLevelMin() == mrHead2.GetDisplayNode().GetWindowLevelMin()}")
print(f"Index 1 window/level max correct: {b.GetWindowLevelMax() == mrHead2.GetDisplayNode().GetWindowLevelMax()}")
print(f"Index 1 LowerThreshold correct: {b.GetLowerThreshold() == mrHead2.GetDisplayNode().GetLowerThreshold()}")
print(f"Index 1 UpperThreshold correct: {b.GetUpperThreshold() == mrHead2.GetDisplayNode().GetUpperThreshold()}")
9

It seems that when sequence support was added to most nodes in Slicer core then vtkMRMLScalarVolumeDisplayNode was accidentally missed. This is why vtkMRMLScalarVolumeDisplayNode nodes do not show up in the proxy node selector in the Sequences module.

The fix is to implement CopyContent method. It would be great if you could work on it.

10

Yes I can look into adding that.

In addition to creating a sequence node of pure vtkMRMLScalarVolumeDisplayNodes, should adding vtkMRMLScalarVolumeNodes as data nodes to a sequence also include copying over their display node if present?

11

I don’t recall experiencing issues myself or reported by users about how display node is initialized now, but I assume that things can be always improved.

12

Recording scalar volume node changes in a sequence is not obviously a reason for automatically recording its display node changes. If I understand your suggestion correctly. E.g. I work with ultrasound sequences. They are replayed often for manual annotations by different users. Each user has different color map preferences and window/level preferences when they work on annotations. If you change the current default behavior, then the display properties recorded during ultrasound recording will be forced on the annotator person by default. We would need to implement a special module that applies the display node set up by the annotator. Or we need to implement a special module that removes display nodes from the recorded nodes during recording.
It seems more intuitive for me to keep the current behavior, when you need to explicitly add the display node to the sequence browser if you want to record display changes too.

13

In this case are users setting a global colormap and window/level that are applied to all frames in the sequence?

In my case, I have optical luminescent images where it is common for different Window/Levels to be used for each frame in the optical sequence. It was confusing for me to add a single frame which had a set of display settings and then upon adding it to a sequence those settings were lost and things reverted back to some default. I was expecting the “state” of the volume node to remain the same when added to a sequence and that includes both the image data and other supporting nodes like the display node.

14

Yes, ultrasound sequences are always viewed with the same window/level (and other display) parameters across the whole sequence. So the same tissues always appear similar, regardless of other brighter or darker areas that may appear in subsequent frames around the tissue. It would be quite confusing if we removed the ultrasound from the patient and we saw a medium gray image (looks like tissue) instead of a black frame (air with no tissue echoes).
I always liked this feature of Slicer to detach volumes nodes from their display nodes, so we can use, record, and replay them separately or together based on what the use case demands.

15

I have issued the following PR to solve the issue of adding a vtkMRMLScalarVolumeDisplayNode to a sequence and it not copying the Window/Level and Threshold states.

2 Likes