# Implementing a custom GPU raycasting function

**URL:** <https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055>\
**Category:** Support\
**Created:** [August 17, 2020, 8:04pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055 "2020-08-17T20:04:37Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zerfox](https://avatars.discourse-cdn.com/v4/letter/z/eb8c5e/32.png) [@Zerfox](https://discourse.slicer.org/u/Zerfox)\
**Post date:** [August 17, 2020, 8:04pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/1 "2020-08-17T20:04:37Z")

</div>

Dear all,

Currently I am working on a 3D Slicer module in which I need to render isosurfaces and silhouettes.  
Right now I am using `vtkOpenGLGPUVolumeRayCastMapper` and setting a `vtkImageData` object as input data. The voxel values are modified continuously.

The isosurfaces generated using `vtkOpenGLGPUVolumeRayCastMapper` are nice, but I need to have more control over the render results. Especially when I want to combine the isosurfaces with silhouettes. I think I need to define my own ray-casting function that can be called by `vtkOpenGLGPUVolumeRayCastMapper` / `vtkGPUVolumeRaycastMapper`. Is something like this possible at this point in time? If so, what would be the best way to approach this or can someone point me to some good resources?

From what I understand, 3D Slicer is using a slightly modified version of VTK version 8.2.0. VTK 9.0 [was supposed to introduce](https://vtk.org/Wiki/VTK/ProgrammableMultiVolumeRendering) shader replacement functionality in `vtkGPUVolumeRaycastMapper`. I am getting a bit lost as the raycasting functionality has received a major overhaul in the past and I still encounter some older documentation while trying to search for an approach to my problem. For instance, for the CPU-based `vtkVolumeRayCastMapper` there appears to have been a `vtkVolumeRayCastFunction` superclass that could have been inherited.

My apologies if this has been asked numerous times or is supposed to be trivial. Getting to know the 3D Slicer and VTK ecosystem is quite overwhelming for me at the moment. It seems that I have difficulties finding the right search keywords.

Thanks in advance.

Kind regards,  
Rutger

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [August 17, 2020, 8:22pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/2 "2020-08-17T20:22:34Z")

</div>

Yes, you should be able to do that now using the PRISMRendering extension which gives you an easy way to work with custom shaders.

> **[ETS-vis-interactive/SlicerPRISMRendering](https://github.com/ETS-vis-interactive/SlicerPRISMRendering)**
>
> Implementation of the PRISM customizable volume rendering framework as a 3D Slicer module - ETS-vis-interactive/SlicerPRISMRendering

> **[PRISM: An open source framework for the interactive design of GPU volume...](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0193636)**
>
> Direct volume rendering has become an essential tool to explore and analyse 3D medical images. Despite several advances in the field, it remains a challenge to produce an image that highlights the anatomy of interest, avoids occlusion of important...

Let us know how it goes for you.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [August 17, 2020, 9:05pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/3 "2020-08-17T21:05:22Z")

</div>

Prism contains nice examples and infrastructure for getting additional inputs from other nodes, but if you just need a custom shader replacements then you can simply set those in the volume rendering node, as shown in these simpler examples: [https://github.com/PerkLab/SlicerSandbox/tree/master/VolumeRenderingSpecialEffects](https://github.com/PerkLab/SlicerSandbox/tree/master/VolumeRenderingSpecialEffects)

---

<div class="post-metadata">

**Author:** ![Zerfox](https://avatars.discourse-cdn.com/v4/letter/z/eb8c5e/32.png) [@Zerfox](https://discourse.slicer.org/u/Zerfox)\
**Post date:** [August 17, 2020, 9:15pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/4 "2020-08-17T21:15:42Z")

</div>

Thanks to both for your fast response! I’ll have a good look and get back to you.

---

<div class="post-metadata">

**Author:** ![Zerfox](https://avatars.discourse-cdn.com/v4/letter/z/eb8c5e/32.png) [@Zerfox](https://discourse.slicer.org/u/Zerfox)\
**Post date:** [August 19, 2020, 7:46am UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/5 "2020-08-19T07:46:35Z")

</div>

I have looked at both provided sources and googled around a bit, but I got stuck again as I am using C++ instead of Python.

@pieper For PRISM I was able to find [the original C++ implementation](https://github.com/drouin-simon/PRISM), which was targeted at VTK 6.3.0. Certain VTK-specific functions are not present in Slicer’s VTK and I am wondering whether it would have worked otherwise. Is there a C++ version of PRISM for Slicer? I was not able to find one. Or should I try to migrate the original PRISM implementation?

@lassoan Similarly, I tried to convert the provided Python example into C++. [On the wiki](https://www.slicer.org/wiki/Documentation/Nightly/Modules/VolumeRendering) I found a piece of code under “How to programmatically volume render your volume node” which was the following:

```
qSlicerAbstractCoreModule* volumeRenderingModule =
  qSlicerCoreApplication::application()->moduleManager()->module("VolumeRendering");
vtkSlicerVolumeRenderingLogic* volumeRenderingLogic = 
  volumeRenderingModule ? vtkSlicerVolumeRenderingLogic::SafeDownCast(volumeRenderingModule->logic()) : 0;
vtkMRMLVolumeNode* volumeNode = mrmlScene->GetNodeByID('vtkMRMLScalarVolumeNode1');
if (volumeRenderingLogic)
  {
  vtkSmartPointer<vtkMRMLVolumeRenderingDisplayNode> displayNode =
    vtkSmartPointer<vtkMRMLVolumeRenderingDisplayNode>::Take(volumeRenderingLogic->CreateVolumeRenderingDisplayNode());
  mrmlScene->AddNode(displayNode);
  volumeNode->AddAndObserveDisplayNodeID(displayNode->GetID());
  volumeRenderingLogic->UpdateDisplayNodeFromVolumeNode(displayNode, volumeNode);
  }

```

Is this outdated code? Because it tries to pass a `vtkMRMLNode*` into a `vtkMRMLVolumeNode*` and a `vtkSmartPointer<vtkMRMLVolumeRenderingDisplayNode>` into `vtkMRMLNode*`, which is not working out of the box. Can I safely cast these? Or do you perhaps have a C++ resources available on custom shader replacements?

Thanks again.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [August 19, 2020, 12:05pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/6 "2020-08-19T12:05:05Z")

</div>

You can safely downcast a vtkMRMLNode to any derived class (search for downcast in Slicer source).

You can convert between Python and C++ code by following a few rules (maybe we have already povided such rules in some previous topic). Therefore, I would recommend to implement a working code snippet in Python first, even if final implentation will be C++, just because it is so much easier to change the code and rerun in Python. You can copy-paste code into the console, create/modify a Python scripted module, or use Jupyter notebook.

---

<div class="post-metadata">

**Author:** ![Zerfox](https://avatars.discourse-cdn.com/v4/letter/z/eb8c5e/32.png) [@Zerfox](https://discourse.slicer.org/u/Zerfox)\
**Post date:** [August 20, 2020, 10:43am UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/7 "2020-08-20T10:43:56Z")

</div>

Thank you for your suggestion to first implement a working code snippet in Python first. I have tried to get the SlicerSandbox module to work properly, after fixing two minor bugs I am running into an error when updating the custom shader (line 288 in VolumeRenderingSpecialEffects.py):

```
AttributeError: 'vtkSlicerVolumeRenderingModuleMRMLPython.vtkMRMLCP' object has no attribute 'GetOrCreateShaderPropertyNode'

```

This error occurs when trying to copy-paste the code into the console, but also when trying to run the module from the GUI. I tried this using the Slicer v4.10.2 binary. When using Slicer built from source (4.11.0-2020-07-09) I do not get this error message, and the volume is raycasted. However, changing the radius for a certain MarkupsFiducial for both sphere and wedge crop doesn’t do anything.

What am I doing wrong?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [August 20, 2020, 8:44pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/8 "2020-08-20T20:44:07Z")

</div>

> [@Zerfox](#):
>
> I tried this using the Slicer v4.10.2 binary

For any developments, use latest Slicer Preview Release.

> [@Zerfox](#):
>
> changing the radius for a certain MarkupsFiducial for both sphere and wedge crop doesn’t do anything

Markups control point size is not related to volume rendering. VolumeRenderingSpecialEffects module serves as an example of how to get additional rendering parameters using a custom module. By using PRISM volume rendering extension, you can create custom shading effects and get GUI widgets to control additional parameters without any Python coding.

---

<div class="post-metadata">

**Author:** ![Zerfox](https://avatars.discourse-cdn.com/v4/letter/z/eb8c5e/32.png) [@Zerfox](https://discourse.slicer.org/u/Zerfox)\
**Post date:** [August 21, 2020, 8:15pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/9 "2020-08-21T20:15:59Z")

</div>

Pardon me for my prior confusion. I finally realise what the purpose for both options are.

> [@lassoan](#):
>
> For any developments, use latest Slicer Preview Release.

Thanks for the remark! As I am normally working in C++ I was using the nightly version of Slicer. I assumed (apparently wrongly) that for Python-related development all binary versions would work.

> [@lassoan](#):
>
> Markups control point size is not related to volume rendering. VolumeRenderingSpecialEffects module serves as an example of how to get additional rendering parameters using a custom module. By using PRISM volume rendering extension, you can create custom shading effects and get GUI widgets to control additional parameters without any Python coding.

Gotcha. I have successfully played around with the code in VolumeRenderingSpecialEffects now too. Thank you for your time and keep up the good work 🙂

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [August 21, 2020, 9:20pm UTC](https://discourse.slicer.org/t/implementing-a-custom-gpu-raycasting-function/13055/10 "2020-08-21T21:20:45Z")

</div>

Keep us posted if you get something working. It would be nice to see what special visualizations you could achieve.