New extension manager and issues with corporate certificates

Can you download the files using Python, in Slicer’s Python environment (using wget, curl, requests,…)?

This seems to work. At least it is finding the filename correct, I do not know how to write

>>> import wget
>>> url = "https://slicer-packages.kitware.com/api/v1/file/613b26b1342a877cb3bee81e/download"
>>> filename = wget.download(url)
>>> filename
'30168-win-amd64-SlicerMorph-gitf70c039-2021-08-20.zip'

OK, this is useful information (the file is saved in the current working directory).

Does the download work using Qt with the code snippet below?

url = qt.QUrl("https://slicer-packages.kitware.com/api/v1/file/613b26b1342a877cb3bee81e/download")
request = qt.QNetworkRequest(url)
manager = qt.QNetworkAccessManager()
reply = manager.get(request)

while (not reply.isFinished()):
    slicer.app.processEvents()

localFile = qt.QFile("c:/tmp/downloaded2.zip")
localFile.open(qt.QIODevice.WriteOnly)
localFile.write(reply.readAll());
localFile.close()

print(f"HTTP response code: {reply.attribute(qt.QNetworkRequest.HttpStatusCodeAttribute)}")
print(f"Error code: {reply.error()}")

There is now a downloaded.zip file in c:/tmp but it is 0 bytes.

>>> localFile = qt.QFile("c:/temp/downloaded.zip")
>>> localFile.open(qt.QIODevice.WriteOnly)
True
>>> localFile.write(reply.readAll());
0
>>> localFile.close()

Sorry, I forgot to say that you need to wait for the request to complete before you read out the result. I’ve updated the code above to wait so that you don’t need to wait manually. Try again with the updated code snippet.

Still 0.


>>> url = qt.QUrl("https://slicer-packages.kitware.com/api/v1/file/613b26b1342a877cb3bee81e/download")
>>> request = qt.QNetworkRequest(url)
>>> manager = qt.QNetworkAccessManager()
>>> reply = manager.get(request)
>>>
>>> while (not reply.isFinished()):
... slicer.app.processEvents()
...
>>> localFile = qt.QFile("c:/temp/downloaded2.zip")
>>> localFile.open(qt.QIODevice.WriteOnly)
True
>>> localFile.write(reply.readAll());
0
>>> localFile.close()
>>>
>>> print(f"HTTP response code: {reply.attribute(qt.QNetworkRequest.HttpStatusCodeAttribute)}")
HTTP response code: None
>>> print(f"Error code: {reply.error()}")
Error code: 6
>>>

Great, you’ve managed to reproduce the problem!

What is the output of reply.errorString()?

‘SSL handshake failed’

Perfect! Now you have everything to debug this problem and find how to fix the SSL configuration to make the handshake happen (by fixing the root cause of the error or making the error ignored). See all the configuration options in the network access manager and the network request.

Can anyone else replicate this issue or suggest what might be the solution? It sounds like this is something to do with the institutional firewall and SSL certificate management.

I won’t know where to start with those.

However, as Steve pointed out I think this is most likely related to certificate. On the same computers, when I install the windows Git, if I do not choose to trust the windows certificate store, git would fail with the same error. A more detailed description of this is here (see the first answer): How do I configure Git to trust certificates from the Windows Certificate Store? - Stack Overflow

Is there a way to test this option for Slicer as well?

Yes, I really appreciate if any Slicer users behind corporate firewalls with self-signed certificates can try this and see if this happens to others as well.

I won’t know where to start with those.

Just Google qt linux "SSL handshake failed" and try all the solutions that people recommend. It should be easy to disable verification, but it would be much nicer if you found out where Python gets the certificates from and make Qt find those, too. But maybe Python (wget) just does not verify the certificates by default.

You may be able to add your self-signed certificate to the certificate file in share\Slicer-4.13\Slicer.crt. See more information here.

OK. Will try those but looking at this example from your link, I am getting this error

 w = slicer.qSlicerWebWidget()
 w.show()
 v = w.webView()
 v.setUrl(qt.QUrl("https://www.eff.org/https-everywhere"))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: QWebEngineView has no attribute named 'setUrl'

We have a very good, simple way to reproduce the error, just stick to that for now (the qSlicerWebWidget contains a full web browser, i.e., an entire operating system, so that could complicate things).

Just check what is needed to make the simple download code snippet work.

For example, check if disabling verification fixes the issue. If it does, then that could be a simple, universal workaround that we could expose, if needed (and if we don’t find a safer solution).

Another idea to try: select different protocol.

Manually adding the certificates inside the Slicer.crt fixed the problem. I can now use the extension manager.

But where do we go from here? This is not a solution for us, just in our center there about 30-40 different installations of Slicer in dozens of computers (each user account needs to be fixed individually). There are many other clinical centers and researchers at SCH who use Slicer that will suffer from this.

@mikebind @ezgimercan do you encounter this certificate issue at your work computers? (try with a new installation)

Try to figure out why wget does not have this issue.

  • Is it because curl can find the certificate on your system? Can you check where self-signed certificates are stored on your computer?
  • Maybe because curl uses the system OpenSSL? You can consult with @jcfr, currently evaluating building Slicer without bundling OpenSSL - see #5663)
  • Is it because SSL verification is turned off? By default it should be enabled, but if your system has a special OpenSSL configuration and the user is not concerned about security then we could allow disabling SSL verification.

Only about 1 of 10 Slicer downloads are on Linux, and they are expected to be able to deal with things like installing self-signed certificates. It would be important to know if this is an issue on Windows and macOS as well.

This happens on both windows and Linux computers that are connected to our network. We are primarily a windows shop, which is the concern. As you said, most Linux users are used to following these kinds of instructions, but quite concerning for windows.

Our windows desktops use a centralized image, in which I presume the certificates are baked. I can try to find path where those are, if that’s helpful.

On Windows, there is a certificate store where you all the trusted certificates are saved to. It would be nice if you could check that Python curl and requests retrieve certificates from there (you can try that for example by temporarily removing your self-signed certification from the store).

By the way, I don’t understand how a large organization cannot afford to pay a few hundred dollars to get the certificate properly registered. This investigation already costs more than what they would need to pay for it. Doesn’t this untrusted certificate cause all kinds of issues for many other applications? How can you connect devices that do not allow installing self-signed certificates?

I have no idea why they are not doing this, and I unfortunately i have no answer for it…