Incorrect volume rendering bounds - first and last slice

Development
1

Summary: volume rendering bounds are incorrect for the first and last slice and not consistent for CPU, GPU, and multi-volume settings

I was getting some wrong bounds on the volume rendering for vtk.js, but it seems to be in slicer as well. The test data to investigate can be found here

I put a recording of this behavior in slicer here

I appreciate any comments

2

Thanks for the report @alireza, I agree the behavior is not consistent and should be improved. As in the vtkjs case it would help to know if this is an application issue or something in the library. It would help to have a pure vtk example to diagnose.

3

Thank you @pieper
I have created a code sandbox with vtk.js only functionalities here with another dataset though.
Iā€™m manually manipulating the first and last slice intensities to be large values for volume rendering to get shown.

4

Excellent, yes. I was also thinking we should have the same thing in python so we can make sure all the renderers are consistent.

5

I created a repo with python implementation here. Iā€™m seeing similar behavior.

I took the example from here and adapted it.

It is using vtkFixedPointVolumeRayCastMapper for the volume mapper. Is there any other type of volume mappers available in vtk python? and what does slicer use?

I wanted to create a google colab, but vtk is not rendering there for some reason (probably rendering)

6

Thanks @alireza, this looks like exactly what we need. Based on this would you be able to file an issue on the VTK (c++) bug tracker?

Slicer uses two mapper types, vtkFixedPointVolumeRayCastMapper and vtkGPUVolumeRayCastMapper, but see this comment about how multi-volume is used. As you showed in your video the three modes are inconsistent and it would be great if they could all have consistently correct behavior.

7

Iā€™ve run into similar issues before due to misinterpreting meaning vtkImageData bounds. It all comes down to how you define ā€œboundsā€ for a VTK data set. It makes things much simpler if bounds are computed as the box that contains all data points. It is very easy to compute this, it works for all data sets (polygonal and volumetric meshes, point sets, etc), and this is the definition that is used throughout VTK. However, developers often donā€™t know or forget that vtkImageData extends 0.5 voxel beyond its ā€œboundsā€ (so GetBounds() output is not the same as the imageā€™s physical bounds), which can lead to seemingly incorrect or inconsistent behavior. Sometimes these can be considered to be bugs, but often you can justify and document the current behavior instead of changing the behavior that so many current software depend on.

Iā€™ve tested with your data set and it seems that all renderers use VTK bounds for rendering (not the physical bounds). Since this has been like this forever (and extrapolating the image might be complicated and/or computationally intensive), I donā€™t think that this will be changed. You can address this by adding a single-voxel boundary (e.g., repeating the border voxels or setting to some uniform value). Considering these, behavior of the 3 raycast mappers in VTK:

  • GPU volume raycast mapper works consistently with all settings.
  • CPU volume raycast mapperā€™s nearest neighbor interpolationā€™s behavior is quite surprising (does not make sense to me) and I would consider it as a bug. You can report this to the VTK bugtracker, but since the mapperā€™s development stopped about 10-15 years ago and probably not many people care about rendering with nearest neighbor ā€œinterpolationā€, probably if you need a fix then you would need to implement it yourself.
  • GPU multivolume raycast mapper indeed computes one side of the bounds incorrectly (it is asymmetric and inconsistent with all other mappers). This mapper is still a work in progress. It has many (more serious) issues. So, it makes sense to report this so that when VTK developers take care of the other issues then they fix this one, too. Submit the sample data set as a mha or non-compressed nrrd image instead of DICOM so that VTK developers can use it more easily.
1 Like
8

Thanks Steve,
Thanks for the links, we had some progress in digging where the problem is coming from in vtk.js and updated the issue on vtk.js repo.
I will file an issue on C++ bug tracker as well

9

Thanks Andras for your comprehensive reply,
Yes, we have noticed the boundary issue as well and I also understand how it will break a lot of things that rely on it if it gets changed

Single-voxel boundary might make sense, but I was hoping whether we can fix this by adding a flag to volume mappers for the exactExtent (?) and default to be false so that it doesnā€™t break things up.

Please let me know your opinion about this

10

Adding a flag (disabled by default) to enable extrapolation beyond the last voxel would be probably OK. This is what is implemented in vtkImageReslice::SetBorder, so I think it would make sense to implement the same options for volume rendering, too.

If you care about half voxels at the boundary then it is important to note that showing something beyond the last voxelā€™s center is extrapolation (if you display interpolated voxel values, either by reslicing or volume rendering). You want to display information that you donā€™t have. You can try to guess what should be there (most common methods are duplication, mirroring, and wraparound) but it may have performance impact and if you make a bad choice then it can cause artifacts or cause quantification errors. So, overall, the safest solution is actually not to add an extrapolated border, for either slice or volume rendering views.

1 Like