Always display annotation or model on top in 3D view

#1

I’m using volume rendering to show a CT volume being cut with a tracked tool. The scene also contains a sugical target, which is represented by a cylinder model and an annotation ruler. Is it possible to show the model and ruler so that they are always visible, even when ‘occluded’ by the rendering?

I’ve attached two images to show what I mean. In the first one the model and annotation ruler are visible because the rendering is clipped ‘behind’ the target. In the second it’s not possible to see the model/ruler becayse the volume is clipped ‘in front’ of the target. I’ve played around with rendering opacity but this does not provide a good visual of the model unless the opacity is very low.

image
image1299×701 182 KB

image
image1271×652 191 KB

#2

@lassoan Any ideas on how to get started on this?

#3

You can get access to the VTK actor and change its rendering layer. Of course, the application does not expect such low-level manipulation, so there might be problems with picking, placing points, etc.

In general, I would avoid such 2.5D view modes and use proper 3D and 2D views instead (you can tune cropping and transparency settings to make sure everything important is visible).

For aligning tool with a trajectory you can show simplified axial and side views of just the tool and the trajectory (bullseye and progress views).

#4

Okay thanks. I know it’s not ideal to do this kind of visualization, but the purpose is actually to compare it to more ‘conventional’ views such as 2D bullseye and progress views which we have already implemented.

From my understanding, 3D views are not very useful for precise navigation but rather rough global navigation (unless you have suggestions for what works well in 3D), so I am trying to make these views more ‘useful’ by augmenting with 2D details such as bullseye or the trajectory projection mentioned in the top post.

Is there a brief description or explanation of how to get access to the vtk actor?

#5

You can get the model displayable manager as described in this example in the script repository, then get vtkActor corresponding to a model display node.

#6

@Prashant_Pandey another approach might be to provide a custom shader that cuts away the part of the volume that is obscuring the annotation. Something like this example: https://www.youtube.com/watch?v=yiEI_yBMu8k (from these experiments)

2 Likes
#7

I got round to trying your suggestions. After getting the vtkActor for the target model display node, I can’t seem to find a way to change the rendering layer.

I also tried modelDisplayManager.GetRenderer().SetLayer(int), but this didn’t effect the 3D view in the expected way.
Any tips?

#8

You need to add a new renderer, set its layer, and move the actor to that new renderer. You may need to update the camera of the new renderer based on the camera of the default layer’s renderer. This is all very fragile but may allow you to confirm that this kind of 2.5-dimensional visualization is clinically useful and worthy of spending time with implementing properly.

#9

I added a new renderer with the modelDisplayNode as an actor, and then played around with the new and old renderer’s layers, but with no luck. I did get some interesting views but not what I was looking for:

image
image701×781 202 KB

#10

You can also display line as 2D actor, which may make it easier to ensure it is above the 3D geometry, without introducing extra renderers or layers.