2021.01.19 Hangout

Tomorrow, we will be having our next weekly hangout at 10:00 AM ET until 11 AM ET.

Anyone is welcome to join to ask questions at https://bit.ly/slicer-googlemeet-hosted-by-kitware

Agenda:

  • Slicer 5 / VTK 9

Feel free to post to this thread to request/suggest a topic!

Thanks
Sam and J-Christophe

I’d like to have a follow up chat about segmentation rendering. I found the vtk shader bug we discussed before so now shading can be made to work (no PR yet, but it’s a small set of changes).

2 Likes

This rendering looks very nice!

It should not be hard to add this as a new volume rendering option. This is required for photorealistic volume rendering (a.k.a. “cinematic rendering”), too - there the RGB array is filled in quickly and approximately using AI-based segmentation, making vessels appear red, bone off-white, soft-tissues brown, etc.

It would be nice to use this in the segmentation displayable manager, too. For this we should have the option of generating opacity volume from the segmentation’s binary labelmap and compute smooth gradients. Smoothing the binary labelmap would be easy, but slow and it would double the memory usage. It would be better to improve the GPU raycaster’s gradient computation to use a small local neighborhood. Eventually, binary labelmap volume rendering as RGB should be a built-in feature of the GPU volume raycast mapper.

Do you use the GPU volume mapper’s labelmap masking features (specify a lookup table for input labelmap mask and mask blend factor controls how much coloring is applied from the mask) or you generate an RGBA volume from scratch?

This one was done by generating the RGBA volume and then rendering withe the GPU ray cast (once the shading was fixed).

Yes, I think the segmentation rendering should all be done in the GPU code.

It would be nice if you could try if setting the segmentation as labelmap mask for the GPU mapper provides the same results. The advantage would be that we would not need to generate an RGBA volume (faster updates, less memory usage).

This is great :pray:

Cc: @Sankhesh_Jhaveri

Yes, exactly.

This image uses the CT as alpha, so you get nice detail with the coloring, but we should also do the option where the segmentation opacity controls the alpha channel. We’ll need to do the local smoothing / surface fitting in the GPU using the segmentation and the color from the lookup table. This should be effectively the same as building a surface model but faster and with some extra options.

I agree we should think about how we should handle volume rendering from a UI perspective.

I would say this is almost a must for this to be a meaningful replacement of existing segmentation rendering, and creating complex visualization from them.

1 Like