Things have observed to break down when Slicer’s SimpleITK is uninstalled a SimpleITK upstream version is installed from whl. Slicer's embedded SimpleITK can be removed by pip · Issue #6711 · Slicer/Slicer · GitHub
As of right now it seems that proceeding with using SimpleITK whl files is not possible within Slicer due to requiring the fixes made in BUG: Update SimpleITK to fix test_sitkUtils by jcfr · Pull Request #6606 · Slicer/Slicer · GitHub which are not in SimpleITK upstream.