New extension manager and issues with corporate certificates

This is because some of the extension icons are server over http.

This will be fixed in the next few weeks.

To help with this, we already developed a Jupyter notebook to identify and check which extensions have issues. The next step will be to harden the notebook and integrate the code into the Slicer/ExtensionsIndex continuous integration.

For reference, see https://github.com/jcfr/jupyter-notebooks/blob/master/45_slicer_extensions_index_check_metadata_urls.ipynb

There is also an automatic URL check implemented in the ExtensionsIndex repository - it just has not been merged yet (it can be used locally, though).

But these http/https warning are not related to the failed download, because I see these warnings on my computer, too, but the download does not fail.

@jcfr is there an update on this? Package download is still failing for me without any detailed error.

This is all there is in the log

Retrieving extension metadata [ extensionId: 61260e2b342a877cb3b8d73e]

Retrieving extension files [ extensionId: 61260e2b342a877cb3b8d73e ]

Downloading extension [ item_id: 61260e2b342a877cb3b8d73e, file_id: 61260e2b342a877cb3b8d745]

Failed downloading: https://slicer-packages.kitware.com/api/v1/file/61260e2b342a877cb3b8d745/download

Here is the screenshot that I shows I can get to the same package via wget on the same laptop.

With todays preview, I tried again and still cannot download packages at office (fine at home).

Retrieving extension metadata [ extensionId: 613b26b1342a877cb3bee817]

Retrieving extension files [ extensionId: 613b26b1342a877cb3bee817 ]

Downloading extension [ item_id: 613b26b1342a877cb3bee817, file_id: 613b26b1342a877cb3bee81e]

Failed downloading: https://slicer-packages.kitware.com/api/v1/file/613b26b1342a877cb3bee81e/download

Can you download the files using Python, in Slicer’s Python environment (using wget, curl, requests,…)?

This seems to work. At least it is finding the filename correct, I do not know how to write

>>> import wget
>>> url = "https://slicer-packages.kitware.com/api/v1/file/613b26b1342a877cb3bee81e/download"
>>> filename = wget.download(url)
>>> filename
'30168-win-amd64-SlicerMorph-gitf70c039-2021-08-20.zip'

OK, this is useful information (the file is saved in the current working directory).

Does the download work using Qt with the code snippet below?

url = qt.QUrl("https://slicer-packages.kitware.com/api/v1/file/613b26b1342a877cb3bee81e/download")
request = qt.QNetworkRequest(url)
manager = qt.QNetworkAccessManager()
reply = manager.get(request)

while (not reply.isFinished()):
    slicer.app.processEvents()

localFile = qt.QFile("c:/tmp/downloaded2.zip")
localFile.open(qt.QIODevice.WriteOnly)
localFile.write(reply.readAll());
localFile.close()

print(f"HTTP response code: {reply.attribute(qt.QNetworkRequest.HttpStatusCodeAttribute)}")
print(f"Error code: {reply.error()}")

There is now a downloaded.zip file in c:/tmp but it is 0 bytes.

>>> localFile = qt.QFile("c:/temp/downloaded.zip")
>>> localFile.open(qt.QIODevice.WriteOnly)
True
>>> localFile.write(reply.readAll());
0
>>> localFile.close()

Sorry, I forgot to say that you need to wait for the request to complete before you read out the result. I’ve updated the code above to wait so that you don’t need to wait manually. Try again with the updated code snippet.

Still 0.


>>> url = qt.QUrl("https://slicer-packages.kitware.com/api/v1/file/613b26b1342a877cb3bee81e/download")
>>> request = qt.QNetworkRequest(url)
>>> manager = qt.QNetworkAccessManager()
>>> reply = manager.get(request)
>>>
>>> while (not reply.isFinished()):
... slicer.app.processEvents()
...
>>> localFile = qt.QFile("c:/temp/downloaded2.zip")
>>> localFile.open(qt.QIODevice.WriteOnly)
True
>>> localFile.write(reply.readAll());
0
>>> localFile.close()
>>>
>>> print(f"HTTP response code: {reply.attribute(qt.QNetworkRequest.HttpStatusCodeAttribute)}")
HTTP response code: None
>>> print(f"Error code: {reply.error()}")
Error code: 6
>>>

Great, you’ve managed to reproduce the problem!

What is the output of reply.errorString()?

‘SSL handshake failed’

Perfect! Now you have everything to debug this problem and find how to fix the SSL configuration to make the handshake happen (by fixing the root cause of the error or making the error ignored). See all the configuration options in the network access manager and the network request.

Can anyone else replicate this issue or suggest what might be the solution? It sounds like this is something to do with the institutional firewall and SSL certificate management.

I won’t know where to start with those.

However, as Steve pointed out I think this is most likely related to certificate. On the same computers, when I install the windows Git, if I do not choose to trust the windows certificate store, git would fail with the same error. A more detailed description of this is here (see the first answer): How do I configure Git to trust certificates from the Windows Certificate Store? - Stack Overflow

Is there a way to test this option for Slicer as well?

Yes, I really appreciate if any Slicer users behind corporate firewalls with self-signed certificates can try this and see if this happens to others as well.

I won’t know where to start with those.

Just Google qt linux "SSL handshake failed" and try all the solutions that people recommend. It should be easy to disable verification, but it would be much nicer if you found out where Python gets the certificates from and make Qt find those, too. But maybe Python (wget) just does not verify the certificates by default.

You may be able to add your self-signed certificate to the certificate file in share\Slicer-4.13\Slicer.crt. See more information here.

OK. Will try those but looking at this example from your link, I am getting this error

 w = slicer.qSlicerWebWidget()
 w.show()
 v = w.webView()
 v.setUrl(qt.QUrl("https://www.eff.org/https-everywhere"))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: QWebEngineView has no attribute named 'setUrl'

We have a very good, simple way to reproduce the error, just stick to that for now (the qSlicerWebWidget contains a full web browser, i.e., an entire operating system, so that could complicate things).

Just check what is needed to make the simple download code snippet work.

For example, check if disabling verification fixes the issue. If it does, then that could be a simple, universal workaround that we could expose, if needed (and if we don’t find a safer solution).

Another idea to try: select different protocol.

Manually adding the certificates inside the Slicer.crt fixed the problem. I can now use the extension manager.

But where do we go from here? This is not a solution for us, just in our center there about 30-40 different installations of Slicer in dozens of computers (each user account needs to be fixed individually). There are many other clinical centers and researchers at SCH who use Slicer that will suffer from this.

@mikebind @ezgimercan do you encounter this certificate issue at your work computers? (try with a new installation)