This is odd. On computers that we tested on, using Intel TBB as SMP backend took care of interference of NVidia threaded optimization (see https://github.com/Slicer/Slicer/pull/930).
Have you built Slicer or extensions yourself? Do you use Intel TBB as SMP backend? (by default, it is used on Windows, but you can double-check the value of Slicer_VTK_SMP_IMPLEMENTATION_TYPE CMake variable).
Does calling vtk.vtkMultiThreader.SetGlobalMaximumNumberOfThreads(1) make any difference?
Maybe you can try to update your NVidia drivers. It would be also nice if you could test on a more recent computer (6th generation Intel CPUs are about 5 years old now, maybe a new model could be significantly faster).