Error While using 'Total Segmentator'

Support
22

@rbumm
Does total segmentator require cuDNN beyond regular cuda installation? I am getting this error message on Ubuntu 22.04 with Slicer 5.2.1. Cuda 12.0 is installed, and pytorch extension installed cu118 enabled torch.

Traceback (most recent call last):
  File "/home/exouser/Slicer/bin/Python/slicer/util.py", line 2961, in tryWithErrorDisplay
    yield
  File "/home/exouser/Slicer/NA-MIC/Extensions-31317/TotalSegmentator/lib/Slicer-5.2/qt-scripted-modules/TotalSegmentator.py", line 258, in onApplyButton
    self.logic.process(self.ui.inputVolumeSelector.currentNode(), self.ui.outputSegmentationSelector.currentNode(),
  File "/home/exouser/Slicer/NA-MIC/Extensions-31317/TotalSegmentator/lib/Slicer-5.2/qt-scripted-modules/TotalSegmentator.py", line 647, in process
    self.logProcessOutput(proc)
  File "/home/exouser/Slicer/NA-MIC/Extensions-31317/TotalSegmentator/lib/Slicer-5.2/qt-scripted-modules/TotalSegmentator.py", line 556, in logProcessOutput
    raise CalledProcessError(retcode, proc.args, output=proc.stdout, stderr=proc.stderr)
subprocess.CalledProcessError: Command '['/home/exouser/Slicer/bin/../bin/PythonSlicer', '/home/exouser/Slicer/lib/Python/bin/TotalSegmentator', '-i', '/tmp/Slicer-exouser/__SlicerTemp__2023-01-14_01+20+25.853/total-segmentator-input.nii', '-o', '/tmp/Slicer-exouser/__SlicerTemp__2023-01-14_01+20+25.853/segmentation', '--ml', '--task', 'total', '--fast']' returned non-zero exit status 1.

and then there is this error on the module pane:

image
image1361×178 60.8 KB

23

Hi

I think you need CUDA 11.6 or 11.7

Hope it helps

24

Thanks I already got the cuda part taken care of. PyTorch worked fine. The issue was indeed the missing cuDNN library, which is not part of the CUDA package and you actually have to register with Nvidia, agree to their terms and download it. https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

I can confirm that after installing it (cuDNN8.7.0), everything it worked fine. It is shame though that Nvidia makes getting cuDNN so hard.

1 Like
25

@muratmaga great that it works now, on my windows systems cudnn and it´s folders seem to be present automatically on every Python where TotalSegmentator is installed, and there has never be the need for separate downloads. Would be interesting to find out why it was necessary on your system …

26

I use Windows, too, and I don’t remember ever having to manually install cuDNN either. Nobody else has reported this issue until now either. So, probably this manual cuDNN installation is only needed on certain Linux systems.

27

I’ve run into this before too. The Nvidia cuDNN instructions clearly say you need to get it via the developer program for both windows and linux: Installation Guide - NVIDIA Docs).

But apparently this is bundled in pytorch: How to check if torch uses cuDNN - #3 by timonbimon - PyTorch Forums

“if you want to use pytorch with an NVIDIA GPU, all you need to do is install pytorch binaries and start using it. We ship with everything in-built (pytorch binaries include CUDA, CuDNN, NCCL, MKL, etc.).”

1 Like
28

@muratmaga is it possible that your CUDA version is not 11.6 or 11.7?

29

Latest Cuda (Cuda 12) is not supported with latest PyTorch binaries.

30

That’s interesting because I used the pytorch extension to install and it clearly brought torch with cuda (11.8) but without the cudnn.

31

@muratmaga Have you installed CUDA? Which version? What CUDA version do you see in pytorch util module in Slicer (cuNNN)?

32

Yes, Cuda 12.0 is installed on this machine. This is what torch reports in Slicer through python:

>>> torch.__version__
'1.13.1+cu117'

(I thought it was 118, but were mistaken).

And this is from pyutils
image

33

A bit more clarification.

I wasn’t aware that pytorch extension (through light the torch) bring all the necessary cuda/cudnn and stuff internally, without requiring the OS based manual installation. I can confirm that this works (and much more simply. yay).

However, if you do have CUDA manually installed and it is path is listed on your LD_LIBRARY_PATH, this seems to interfere with the pyTorch util detection, and you will need to provide the cuDNN manually.

Sorry for the confusion.

34

Thanks for the clarification. Do you mean that on linux you don’t event need to install CUDA, but only the NVIDIA drivers?

35

Yep! That’s what the link @pieper cited said, and I can confirm it is indeed the case.

1 Like
36

3 posts were split to a new topic: Use AMD ROCm for TotalSegmentator

37

A post was merged into an existing topic: Total segmentor module showing errors