I would suggest to use ASCII filenames, as it is done in all the millions of Python packages out there.
There are problems with utf8 on Windows with latest stable because we switched to utf8 code page in the application so that internally we can use utf8 everywhere. However, the Windows console and/or Python 3.6 has limited support for utf8 application code page.
If this is important for you then please investigate. For example, try building Slicer with Python 3.8 or later or ask around on Python forums about how to embed Python in applications that use utf8 code page.