Stitching artifact with volume rendering large volumes (Feb 17 2021 nightly)

Development
1

I’m running into a weird stitching artifact with volume rendering, and at first I thought it was my volume (rather large) but I can duplicate it by resampling available sample data to make a large volume.

This is on a 2019 MacBook Pro 16" with a AMD Radeon Pro 5500M 8 GB, using GPU Raycasting.

  1. Start slicer
  2. Download CT Chest sample data
  3. Go to Crop Volume
  4. Create an ROI, fit to volume
  5. In spacing scale, pick 0.3x, so new volume is 1706x1706x483
  6. Click “Apply”
  7. Go to data module, delete original volume (save space)
  8. Go to volume rendering, click eye to make volume visible

It looks like the volume is split into several blocks, where the data looks like it’s wrapping or something. CPU recasting works (but is slow, of course). No errors in the console. I do not think I’m violating AMD’s 3D texture limit of 2048.

Image below. Any ideas?

Thanks!!

-Hollister

image
image1590×1094 460 KB

2

Which Slicer version is this? Do you see any errors or warnings in the log?

3

4.13.0-2021-02-17, the MacOS Feb 17 nightly build. No errors or warnings in the log.

4

Incidentally, I made a dataset that exceeds the AMD 3D texture size limit and there are no errors in the log. So perhaps this is what is happening? @muratmaga, you mentioned this a while back on a thread about external GPUs. Have you run into instances where you’ve exceeded AMD’s 3D texture limit?

Thanks!

5

It works fine for me with an NVidia RTX2080 on Windows (other than the OS sometimes shutting down the application from time to time due to TDR timeout; but I could increase the timeout to prevent this). Most likely the problem you are experiencing is is a driver bug or hardware limitation.

6

I’ve been considering an external GPU with 16GB of graphics card memory, but not if I still run into this problem. Need to find a Mac somewhere with 16GB graphics memory to test this.

Thanks for checking! Much appreciated.

-Hollister

7

Apologies for the cross-post from an earlier thread on external graphics cards.

I’m not so sure this is an issue with the maximum 3D texture size on AMD cards. OpenGL caps viewer says this is 16384 on my system.

Doesn’t answer what’s going on, however.

8

Neither did I have any issue on Linux with Geforce 1080TI

I only have the TDR, when I have the GPU quality set to adaptive. Even on 1080TI the speed is great when the quality set to Normal (in adaptive it crashes).

9

I’ve checked this and the same happens on my computer. “Maximum quality” mode causes TDR timeout immediately, “Adaptive” mode works fine if I rotate but times out when for example I center the view. “Adaptive” mode adjusts sampling distance to higher/lower values to determine the maximum image quality that can be achieved on the GPU and during these iterations it can run into slower rendering times, which can cause the TDR timeout.

Since rendering time estimation for GPU is very complex, it is hard to come up with a good adaptive scheme, so using “Normal” mode is reasonable.

10

Down-sampling my large volume from 16 bit to 8 bit makes the problem go away. So it’s not an issue with image dimensions, but of memory usage. (I think.)

11

Also from other thread - I cropped and resampled MRHead to be like 200x200x4096, and it renders fine. So it’s most likely memory usage (somehow).

12

I was able to load the original volume that’s been giving me problems (16 bit, 922x1132x2020) into an older version of Drishti, which also does volume rendering using OpenGL but is much more limited in segmentation capabilities, and it renders fine with no stitching. So it is not a MacOs or OpenGL issue.

ParaView has the same issue when I load the NRRD file in. Could this be a MacOS ITK issue?

13

Make that VTK, not ITK

14

Check the source code of drishti (GitHub - nci/drishti: Drishti). If its rendering code base is old then it is very likely that it downscales the volume before sending it to the GPU, because 8-10 years ago it was very rare that you could render a volume at its native resolution. VTK used to do this in its previous rendering backend, but this feature has not been reimplemented yet after its rendering backend was reworked a few years ago.

It is of course also possible that there is a bug in VTK how it splits up and sends the texture to the GPU (as far as I remember it does something like that), so I would recommend to write about this to the VTK forum. Maybe somebody recognizes the appearance of this artifact. It is good that it is reproducible in Paraview, too, because it proves that the problem is not how Slicer uses VTK.

15

Okie dokie, thanks for the tips. Drishti for Mac hasn’t been updated in some time so it wouldn’t surprise me if it downsampled. It was only in the last update that it supported 16 bit data, for example.

I’ve been messing with some of the python VTK examples to get a simple example that shows the problem. But yeah, the fact that ParaView does it too is a useful data point.

Thanks, Andras!

16

One more thing to try. You can partition the volume (upload to GPU in smaller blocks) using SetPartition method. For example, you can try this script (or change partitions to 4,4,4 or 1,1,2, …) and see it if makes any difference:

threeDViewWidget = slicer.app.layoutManager().threeDWidget(0)
vrDisplayableManager = threeDViewWidget.threeDView().displayableManagerByClassName('vtkMRMLVolumeRenderingDisplayableManager')
vrMapper = vrDisplayableManager.GetVolumeMapper(getNode('CTChest'))
vrMapper.SetPartitions(2,2,2)

Overall, I don’t see much difference when using partitions, but SetPartition(4,4,4) seems to decrease TDR occurrence a bit for me (that otherwise would quite frequently occur with the “Adaptive” quality option).

1 Like
17

Hmm, this seems to do something, or at least have some sort of effect. Screen shot is with 2,2,2 - I will play with this some more and see what it does.

Thanks!

image
image624×534 142 KB

18

OKAY! (1,1,2) seems to work!

THANKS A MILLION!!

image
image561×554 110 KB

19

Awesome!

@muratmaga please keep an eye on this. If partitions can fix rendering of large volumes then we may expose this option in the GUI (or if we learn more about what settings work then even enable it automatically above a certain volume size).

20

OK. I will unfortunately, i never encountered this issue on the nvidia cards we have…

1 Like