Implementing a custom GPU raycasting function

Support
1

Dear all,

Currently I am working on a 3D Slicer module in which I need to render isosurfaces and silhouettes.
Right now I am using vtkOpenGLGPUVolumeRayCastMapper and setting a vtkImageData object as input data. The voxel values are modified continuously.

The isosurfaces generated using vtkOpenGLGPUVolumeRayCastMapper are nice, but I need to have more control over the render results. Especially when I want to combine the isosurfaces with silhouettes. I think I need to define my own ray-casting function that can be called by vtkOpenGLGPUVolumeRayCastMapper / vtkGPUVolumeRaycastMapper. Is something like this possible at this point in time? If so, what would be the best way to approach this or can someone point me to some good resources?

From what I understand, 3D Slicer is using a slightly modified version of VTK version 8.2.0. VTK 9.0 was supposed to introduce shader replacement functionality in vtkGPUVolumeRaycastMapper. I am getting a bit lost as the raycasting functionality has received a major overhaul in the past and I still encounter some older documentation while trying to search for an approach to my problem. For instance, for the CPU-based vtkVolumeRayCastMapper there appears to have been a vtkVolumeRayCastFunction superclass that could have been inherited.

My apologies if this has been asked numerous times or is supposed to be trivial. Getting to know the 3D Slicer and VTK ecosystem is quite overwhelming for me at the moment. It seems that I have difficulties finding the right search keywords.

Thanks in advance.

Kind regards,
Rutger

2

Yes, you should be able to do that now using the PRISMRendering extension which gives you an easy way to work with custom shaders.

Let us know how it goes for you.

3

Prism contains nice examples and infrastructure for getting additional inputs from other nodes, but if you just need a custom shader replacements then you can simply set those in the volume rendering node, as shown in these simpler examples: https://github.com/PerkLab/SlicerSandbox/tree/master/VolumeRenderingSpecialEffects

4

Thanks to both for your fast response! I’ll have a good look and get back to you.

5

I have looked at both provided sources and googled around a bit, but I got stuck again as I am using C++ instead of Python.

@pieper For PRISM I was able to find the original C++ implementation, which was targeted at VTK 6.3.0. Certain VTK-specific functions are not present in Slicer’s VTK and I am wondering whether it would have worked otherwise. Is there a C++ version of PRISM for Slicer? I was not able to find one. Or should I try to migrate the original PRISM implementation?

@lassoan Similarly, I tried to convert the provided Python example into C++. On the wiki I found a piece of code under “How to programmatically volume render your volume node” which was the following:

qSlicerAbstractCoreModule* volumeRenderingModule =
  qSlicerCoreApplication::application()->moduleManager()->module("VolumeRendering");
vtkSlicerVolumeRenderingLogic* volumeRenderingLogic = 
  volumeRenderingModule ? vtkSlicerVolumeRenderingLogic::SafeDownCast(volumeRenderingModule->logic()) : 0;
vtkMRMLVolumeNode* volumeNode = mrmlScene->GetNodeByID('vtkMRMLScalarVolumeNode1');
if (volumeRenderingLogic)
  {
  vtkSmartPointer<vtkMRMLVolumeRenderingDisplayNode> displayNode =
    vtkSmartPointer<vtkMRMLVolumeRenderingDisplayNode>::Take(volumeRenderingLogic->CreateVolumeRenderingDisplayNode());
  mrmlScene->AddNode(displayNode);
  volumeNode->AddAndObserveDisplayNodeID(displayNode->GetID());
  volumeRenderingLogic->UpdateDisplayNodeFromVolumeNode(displayNode, volumeNode);
  }

Is this outdated code? Because it tries to pass a vtkMRMLNode* into a vtkMRMLVolumeNode* and a vtkSmartPointer<vtkMRMLVolumeRenderingDisplayNode> into vtkMRMLNode*, which is not working out of the box. Can I safely cast these? Or do you perhaps have a C++ resources available on custom shader replacements?

Thanks again.

6

You can safely downcast a vtkMRMLNode to any derived class (search for downcast in Slicer source).

You can convert between Python and C++ code by following a few rules (maybe we have already povided such rules in some previous topic). Therefore, I would recommend to implement a working code snippet in Python first, even if final implentation will be C++, just because it is so much easier to change the code and rerun in Python. You can copy-paste code into the console, create/modify a Python scripted module, or use Jupyter notebook.

7

Thank you for your suggestion to first implement a working code snippet in Python first. I have tried to get the SlicerSandbox module to work properly, after fixing two minor bugs I am running into an error when updating the custom shader (line 288 in VolumeRenderingSpecialEffects.py):

AttributeError: 'vtkSlicerVolumeRenderingModuleMRMLPython.vtkMRMLCP' object has no attribute 'GetOrCreateShaderPropertyNode'

This error occurs when trying to copy-paste the code into the console, but also when trying to run the module from the GUI. I tried this using the Slicer v4.10.2 binary. When using Slicer built from source (4.11.0-2020-07-09) I do not get this error message, and the volume is raycasted. However, changing the radius for a certain MarkupsFiducial for both sphere and wedge crop doesn’t do anything.

What am I doing wrong?

8

For any developments, use latest Slicer Preview Release.

Markups control point size is not related to volume rendering. VolumeRenderingSpecialEffects module serves as an example of how to get additional rendering parameters using a custom module. By using PRISM volume rendering extension, you can create custom shading effects and get GUI widgets to control additional parameters without any Python coding.

9

Pardon me for my prior confusion. I finally realise what the purpose for both options are.

Thanks for the remark! As I am normally working in C++ I was using the nightly version of Slicer. I assumed (apparently wrongly) that for Python-related development all binary versions would work.

Gotcha. I have successfully played around with the code in VolumeRenderingSpecialEffects now too. Thank you for your time and keep up the good work :slight_smile:

1 Like
10

Keep us posted if you get something working. It would be nice to see what special visualizations you could achieve.