There are many ways to improve the performance. But first you have to understand the memory usage during segmentation. If your volume is 11GB in size, during the segmentation (depending on the effect), Slicer may use 6-10 times more (so 60-100GB) transiently. That’s already at the limit of available memory on your computer. Adding another segment or five will increase even further. So while your computer is fairly powerful, your dataset is also quite big.
There are many ways to deal with that. For example, it is possible that while your full dataset is huge, perhaps you are only segmenting a portion of it (e.g., skull from a whole fish). Then all your operations will run faster, if you first crop your volume to the area you are going to segment (or only import that section).
As for the slowness during the save/exporting, that’s because Slicer by default compresses the data. Compression of 11GB file does take a while because it is a single threaded task. So to benefit from the fast NVMe disk, disable compression when saving/exporting (if you are using SlicerMorph you can enable the SlicerMorph customizations, which will disable the compression automatically for volume objects).
Your GPU is powerful enough that volume rendering should not be a problem. But volume rendering is not used during segmentation. Show3D button creates polydata (aka 3D model). That may take sometime for large dataset. To improve its performance, you can choose use new 3D model generation algorithm -Surface Nets, and the faster smoothing options).
You should really check the SlicerMorph tutorials we have for segmentation to understand the types of issues you are facing and how to solve them: