Sandbox module cannot be downloaded via extension manager today?

Support
21

After following up with our hospital IT, the problem was related to certificates. As a preferably temporary solution, they added an exception to SSL inspection for slicer-packages.kitware.com. This resolved the issue and enabled downloading of extensions via the extension manager.

However, they did ask me to investigate adding a certificate to Slicer so that they could remove this exception. The discussions in this thread seem quite relevant:

but I have to admit I don’t really follow how this all works. If I go to slicer.org (as an example domain) in a browser and check the certificate viewer, the “Issued By” section says:
image

This was also the case for slicer-packages.kitware.com before the exception was added, and, if I am understanding correctly, it was the absence of this Zscaler certificate in the Slicer crt file which led to the failed download. Zscaler is the software installed on hospital issued laptops for use outside the hospital network so that they can behave as if they are inside the hospital network (I would call this a VPN but since we’re actually talking about details of networking and security I don’t want to accidentally misuse a technical term).

I’d love to find a solution for this which both allowed the hospital to remove the exception for slicer-packages.kitware.com in their firewall AND allowed Slicer users inside the network to use the extension manager normally. I can try to rebuild Slicer.crt with an additional Zscaler certificate added following the thread linked above, but I don’t yet understand how to do that (and, as @muratmaga pointed out, this isn’t a great general solution for clinical Slicer users who may not be technically savvy). On the other thread, there is some discussion of “self-signed” certificates, but I am not sure how to tell if this certificate is self-signed or not. The hospital IT person I worked with was competent and helpful, and I think would be willing to help me troubleshoot further if I knew what questions to be asking or what solutions to be trying.

I find some potentially helpful information on this page: Choosing the CA Certificate for SSL Inspection | Zscaler. I am guessing the hospital is using the first option, SSL Inspection Using Zscaler’s default intermediate CA certificate, which looks like it uses a dynamically generated and signed certificate.

Here are some options for moving forward:

  • explore turning off ssl verification from the Slicer side of the connection so that Slicer does not cancel the download because it doesn’t recognize the zscaler certificate
  • explore adding the zscaler certificate to Slicer’s set of trusted certificates
  • explore having Slicer get the set of trusted certificates from the OS (or something like that) so that it will trust the zscaler certificate the same way the browser does
  • explore pushing to have the zscaler certificate added to the outside trusted repository that Slicer builds its list of trusted certificates from (looks like maybe mozilla?). I think maybe this is essentially what is meant by having it not be self-signed? However, if the certificate is being dynamically generated and signed and involves intercepting and inspecting traffic (see zscaler link above), this approach probably isn’t possible.

Any suggestions about which of these I should pursue? @lassoan

22

Great detective work! I think this explains the all the issues that you have been experiencing.

It seems that ZScaler implements a man-in-the-middle attack on all external network communications. According this blog post ZScaler does not generate generally trustable certificates, just some temporary throwaway ones. The ZScaler agent running on the user’s computer receives these temporary certificates and regularly replaces them on the system.

I’m very surprised that hospitals go this far and doubt that this would be effective (if someone wants to hide something it is always possible to add yet another layer of encryption), while it surely introduces additional complexity and therefore extra cost and more vulnerabilities.

The issue may be due to the Slicer application not trusting the unknown ZScaler certificate but the man-in-the-middle attack might also interfere with the communication between the frontend and backend servers. Adding new certificates in Slicer application should not be hard, so if that is sufficient then we shoul definitely look into it. Adding exceptions or special certificates to the servers is not an option.

Based on these, comments on potential next steps:

explore turning off ssl verification from the Slicer side of the connection so that Slicer does not cancel the download because it doesn’t recognize the zscaler certificate

Turning off SSL verification would be quite risky, as man-in-the-middle attacks could allow malicious actors to get manipulated packages installed on user’s computers.

explore adding the zscaler certificate to Slicer’s set of trusted certificates

It would be interesting to try and see if adding the current ZScaler certificate to Slicer’s .crt file fixes extension installation using the “Manage” tag using bookmarks (in this case Slicer directly communnicates with the extension server backend) and/or installation by using the “Install” tab (in this case the extensions server frontend is involved as well).

explore having Slicer get the set of trusted certificates from the OS (or something like that) so that it will trust the zscaler certificate the same way the browser does

If the previous experiment is successful then we should explore this. Accessing the certificate store is depending on the operating system, but Qt or Python may provide some cross-platform API.

@jcfr Do you know why Slicer brings its own .crt file (instead of trying to get the certificates from the operating system)?

explore pushing to have the zscaler certificate added to the outside trusted repository that Slicer builds its list of trusted certificates from (looks like maybe mozilla?).

Since these are temporary certificates, these cannot be added to any repository (by the time they were added they could be already replaced).

I’ve found many links that complains about ZScaler breaking TLS in various applications and many solutions. A few useful links:

23

Thanks for the helpful guidance! It sounds like the first thing for me to try is to add the ZScaler certificate to the Slicer .crt file. As of right now, I don’t know how to obtain the ZScaler certificate or how to add it to the Slicer .crt file.

I see the readme at Slicer/Base/QTCore/Resources/Certs at main · Slicer/Slicer · GitHub and see how the documentation there guides one to building Slicer.crt from data at https://raw.githubusercontent.com/mozilla/gecko-dev/master/security/nss/lib/ckfw/builtins/certdata.txt. Entries in that file look, for example, like this

# Trust for "Entrust.net Premium 2048 Secure Server CA"
# Issuer: CN=Entrust.net Certification Authority (2048),OU=(c) 1999 Entrust.net Limited,OU=www.entrust.net/CPS_2048 incorp. by ref. (limits liab.),O=Entrust.net
# Serial Number: 946069240 (0x3863def8)
# Subject: CN=Entrust.net Certification Authority (2048),OU=(c) 1999 Entrust.net Limited,OU=www.entrust.net/CPS_2048 incorp. by ref. (limits liab.),O=Entrust.net
# Not Valid Before: Fri Dec 24 17:50:51 1999
# Not Valid After : Tue Jul 24 14:15:12 2029
# Fingerprint (SHA-256): 6D:C4:71:72:E0:1C:BC:B0:BF:62:58:0D:89:5F:E2:B8:AC:9A:D4:F8:73:80:1E:0C:10:B9:C8:37:D2:1E:B1:77
# Fingerprint (SHA1): 50:30:06:09:1D:97:D4:F5:AE:39:F7:CB:E7:92:7D:7D:65:2D:34:31
CKA_CLASS CK_OBJECT_CLASS CKO_NSS_TRUST
CKA_TOKEN CK_BBOOL CK_TRUE
CKA_PRIVATE CK_BBOOL CK_FALSE
CKA_MODIFIABLE CK_BBOOL CK_FALSE
CKA_LABEL UTF8 "Entrust.net Premium 2048 Secure Server CA"
CKA_CERT_SHA1_HASH MULTILINE_OCTAL
\120\060\006\011\035\227\324\365\256\071\367\313\347\222\175\175
\145\055\064\061
END
CKA_CERT_MD5_HASH MULTILINE_OCTAL
\356\051\061\274\062\176\232\346\350\265\367\121\264\064\161\220
END
CKA_ISSUER MULTILINE_OCTAL
\060\201\264\061\024\060\022\006\003\125\004\012\023\013\105\156
\164\162\165\163\164\056\156\145\164\061\100\060\076\006\003\125
\004\013\024\067\167\167\167\056\145\156\164\162\165\163\164\056
\156\145\164\057\103\120\123\137\062\060\064\070\040\151\156\143
\157\162\160\056\040\142\171\040\162\145\146\056\040\050\154\151
\155\151\164\163\040\154\151\141\142\056\051\061\045\060\043\006
\003\125\004\013\023\034\050\143\051\040\061\071\071\071\040\105
\156\164\162\165\163\164\056\156\145\164\040\114\151\155\151\164
\145\144\061\063\060\061\006\003\125\004\003\023\052\105\156\164
\162\165\163\164\056\156\145\164\040\103\145\162\164\151\146\151
\143\141\164\151\157\156\040\101\165\164\150\157\162\151\164\171
\040\050\062\060\064\070\051
END
CKA_SERIAL_NUMBER MULTILINE_OCTAL
\002\004\070\143\336\370
END
CKA_TRUST_SERVER_AUTH CK_TRUST CKT_NSS_TRUSTED_DELEGATOR
CKA_TRUST_EMAIL_PROTECTION CK_TRUST CKT_NSS_TRUSTED_DELEGATOR
CKA_TRUST_CODE_SIGNING CK_TRUST CKT_NSS_MUST_VERIFY_TRUST
CKA_TRUST_STEP_UP_APPROVED CK_BBOOL CK_FALSE

I assume that what I need is the corresponding entry values for the ZScaler certificate. Then it looks like I could run make-ca.sh to generate Slicer.crt. Alternatively, it looks like I could just edit Slicer.crt directly; entries there look like:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1246989352 (0x4a538c28)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = "Entrust, Inc.", OU = See www.entrust.net/legal-terms, OU = "(c) 2009 Entrust, Inc. - for authorized use only", CN = Entrust Root Certification Authority - G2
        Validity
            Not Before: Jul  7 17:25:54 2009 GMT
            Not After : Dec  7 17:55:54 2030 GMT
        Subject: C = US, O = "Entrust, Inc.", OU = See www.entrust.net/legal-terms, OU = "(c) 2009 Entrust, Inc. - for authorized use only", CN = Entrust Root Certification Authority - G2
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:ba:84:b6:72:db:9e:0c:6b:e2:99:e9:30:01:a7:
                    76:ea:32:b8:95:41:1a:c9:da:61:4e:58:72:cf:fe:
                    f6:82:79:bf:73:61:06:0a:a5:27:d8:b3:5f:d3:45:
                    4e:1c:72:d6:4e:32:f2:72:8a:0f:f7:83:19:d0:6a:
                    80:80:00:45:1e:b0:c7:e7:9a:bf:12:57:27:1c:a3:
                    68:2f:0a:87:bd:6a:6b:0e:5e:65:f3:1c:77:d5:d4:
                    85:8d:70:21:b4:b3:32:e7:8b:a2:d5:86:39:02:b1:
                    b8:d2:47:ce:e4:c9:49:c4:3b:a7:de:fb:54:7d:57:
                    be:f0:e8:6e:c2:79:b2:3a:0b:55:e2:50:98:16:32:
                    13:5c:2f:78:56:c1:c2:94:b3:f2:5a:e4:27:9a:9f:
                    24:d7:c6:ec:d0:9b:25:82:e3:cc:c2:c4:45:c5:8c:
                    97:7a:06:6b:2a:11:9f:a9:0a:6e:48:3b:6f:db:d4:
                    11:19:42:f7:8f:07:bf:f5:53:5f:9c:3e:f4:17:2c:
                    e6:69:ac:4e:32:4c:62:77:ea:b7:e8:e5:bb:34:bc:
                    19:8b:ae:9c:51:e7:b7:7e:b5:53:b1:33:22:e5:6d:
                    cf:70:3c:1a:fa:e2:9b:67:b6:83:f4:8d:a5:af:62:
                    4c:4d:e0:58:ac:64:34:12:03:f8:b6:8d:94:63:24:
                    a4:71
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Subject Key Identifier:
                6A:72:26:7A:D0:1E:EF:7D:E7:3B:69:51:D4:6C:8D:9F:90:12:66:AB
    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
        79:9f:1d:96:c6:b6:79:3f:22:8d:87:d3:87:03:04:60:6a:6b:
        9a:2e:59:89:73:11:ac:43:d1:f5:13:ff:8d:39:2b:c0:f2:bd:
        4f:70:8c:a9:2f:ea:17:c4:0b:54:9e:d4:1b:96:98:33:3c:a8:
        ad:62:a2:00:76:ab:59:69:6e:06:1d:7e:c4:b9:44:8d:98:af:
        12:d4:61:db:0a:19:46:47:f3:eb:f7:63:c1:40:05:40:a5:d2:
        b7:f4:b5:9a:36:bf:a9:88:76:88:04:55:04:2b:9c:87:7f:1a:
        37:3c:7e:2d:a5:1a:d8:d4:89:5e:ca:bd:ac:3d:6c:d8:6d:af:
        d5:f3:76:0f:cd:3b:88:38:22:9d:6c:93:9a:c4:3d:bf:82:1b:
        65:3f:a6:0f:5d:aa:fc:e5:b2:15:ca:b5:ad:c6:bc:3d:d0:84:
        e8:ea:06:72:b0:4d:39:32:78:bf:3e:11:9c:0b:a4:9d:9a:21:
        f3:f0:9b:0b:30:78:db:c1:dc:87:43:fe:bc:63:9a:ca:c5:c2:
        1c:c9:c7:8d:ff:3b:12:58:08:e6:b6:3d:ec:7a:2c:4e:fb:83:
        96:ce:0c:3c:69:87:54:73:a4:73:c2:93:ff:51:10:ac:15:54:
        01:d8:fc:05:b1:89:a1:7f:74:83:9a:49:d7:dc:4e:7b:8a:48:
        6f:8b:45:f6
SHA1 Fingerprint=8C:F4:27:FD:79:0C:3A:D1:66:06:8D:E8:1E:57:EF:BB:93:22:72:D4
-----BEGIN CERTIFICATE-----
MIIEPjCCAyagAwIBAgIESlOMKDANBgkqhkiG9w0BAQsFADCBvjELMAkGA1UEBhMC
VVMxFjAUBgNVBAoTDUVudHJ1c3QsIEluYy4xKDAmBgNVBAsTH1NlZSB3d3cuZW50
cnVzdC5uZXQvbGVnYWwtdGVybXMxOTA3BgNVBAsTMChjKSAyMDA5IEVudHJ1c3Qs
IEluYy4gLSBmb3IgYXV0aG9yaXplZCB1c2Ugb25seTEyMDAGA1UEAxMpRW50cnVz
dCBSb290IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzIwHhcNMDkwNzA3MTcy
NTU0WhcNMzAxMjA3MTc1NTU0WjCBvjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUVu
dHJ1c3QsIEluYy4xKDAmBgNVBAsTH1NlZSB3d3cuZW50cnVzdC5uZXQvbGVnYWwt
dGVybXMxOTA3BgNVBAsTMChjKSAyMDA5IEVudHJ1c3QsIEluYy4gLSBmb3IgYXV0
aG9yaXplZCB1c2Ugb25seTEyMDAGA1UEAxMpRW50cnVzdCBSb290IENlcnRpZmlj
YXRpb24gQXV0aG9yaXR5IC0gRzIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC6hLZy254Ma+KZ6TABp3bqMriVQRrJ2mFOWHLP/vaCeb9zYQYKpSfYs1/T
RU4cctZOMvJyig/3gxnQaoCAAEUesMfnmr8SVycco2gvCoe9amsOXmXzHHfV1IWN
cCG0szLni6LVhjkCsbjSR87kyUnEO6fe+1R9V77w6G7CebI6C1XiUJgWMhNcL3hW
wcKUs/Ja5CeanyTXxuzQmyWC48zCxEXFjJd6BmsqEZ+pCm5IO2/b1BEZQvePB7/1
U1+cPvQXLOZprE4yTGJ36rfo5bs0vBmLrpxR57d+tVOxMyLlbc9wPBr64ptntoP0
jaWvYkxN4FisZDQSA/i2jZRjJKRxAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAP
BgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRqciZ60B7vfec7aVHUbI2fkBJmqzAN
BgkqhkiG9w0BAQsFAAOCAQEAeZ8dlsa2eT8ijYfThwMEYGprmi5ZiXMRrEPR9RP/
jTkrwPK9T3CMqS/qF8QLVJ7UG5aYMzyorWKiAHarWWluBh1+xLlEjZivEtRh2woZ
Rkfz6/djwUAFQKXSt/S1mja/qYh2iARVBCuch38aNzx+LaUa2NSJXsq9rD1s2G2v
1fN2D807iDginWyTmsQ9v4IbZT+mD12q/OWyFcq1rca8PdCE6OoGcrBNOTJ4vz4R
nAuknZoh8/CbCzB428Hch0P+vGOaysXCHMnHjf87ElgI5rY97HosTvuDls4MPGmH
VHOkc8KT/1EQrBVUAdj8BbGJoX90g5pJ19xOe4pIb4tF9g==
-----END CERTIFICATE-----

Those seem like they have a relatively straightforward structure, so I probably just need to know how to obtain the information for the current ZScaler certificate. Is there an easy way to do that?

24

Since ZScaler hijacks every site, of you open any site on your web browser and click on the site information button in the browser then you will get the ZScaler certificate. You should be able to copy or export the certification from the browser that can be imported into the crt file.

You could also experent if you can use some Qt API (such as QSslConfiguration Class | Qt Network 6.6.1) to retrieve some default system certificates.

25

A simple solution that I have been using is to turn off zscaler when I need to run Slicer.

26

Thanks for the investigation into this. My group has also been struggling with this issue of installing Slicer extensions. We are not part of a hospital network, but we are part of a large enterprise Life Sciences company. Our company-issued laptops have ZScaler on them which is active when we are home and not connected to the corporate network. The issue of installing Slicer extension is not a problem when I’m in the office and ZScaler is not active.

I must say I am a bit confused about what to do next about the ZScaler cert. Navigating all the menus in Chrome is quite a task to figure out. This information should then be added to the file at the “…\Slicer 5.6.1\share\Slicer-5.6\Slicer.crt” location?

27

@jamesobutler can’t you disable the Zscaler when you are outside of campus and not actively accessing their intranet? For me that solution works.

@mikebind I think your original issue what the lack of proper certificates on the computer itself. Now that’s taken care of, do try turning off the zscaler (you shouldn’t need zscaler when you are on campus anyways). I have been using that method for 1.5 years successfully.

28

I don’t think so. No changes were made on my computer that I am aware of. Instead, I believe what happened was that slicer-packages.kitware.com was added to a whitelist of domains which are not run through the Zscaler inspection at the institution level. When Zscaler is on, nearly all network traffic is routed through it, as can be confirmed by checking the certificate hierarchy in a browser visiting a random web page, everything has a Zscaler certificate. However, after the change, the kitware package site no longer has a Zscaler certificate in the hierarchy. Based on this information, I think you are probably correct that turning Zscaler off may be a viable strategy for installing extensions. However, when I tried it just now, I get the same type of error in Slicer, and I still see Zscaler certificates in the browser, so I think that there is maybe some lingering of settings (maybe some of this stuff is cached?). Restarting Slicer didn’t change anything, and I haven’t tried restarting the computer yet. I work from home nearly all the time, and use the intranet over Zscaler nearly all the time. It is worth it to me to spend some troubleshooting effort to see if it’s possible to resolve this so that it “just works”.

@lassoan , I am still unclear about how to modify the Slicer.crt. As you suggest, in Chrome, on a site, I can get to a button to export a certificate, and that generates a .crt file, but the only thing in it is the block of characters between —BEGIN CERTIFICATE— and —END CERTIFICATE— tags. Is it sufficient to paste that at the bottom of the Slicer.crt file? Or is the metadata section which seems to be present for all other certificates (the version, the serial number, signature algorthim, etc) also required? Also, there seem to be multiple levels of certificates in a hierarchy, do I need to add all of them?

I can no longer use the extension manager for testing, since the exception is now in place allowing that domain to skip zscaler. So, I have been using discourse.slicer.org as the test url instead, which still gives an SSL handshake error.

def testURL(url):
  request = qt.QNetworkRequest(url)
  manager = qt.QNetworkAccessManager()
  reply = manager.get(request)

  while (not reply.isFinished()):
    slicer.app.processEvents()
	
  print(f"HTTP response code: {reply.attribute(qt.QNetworkRequest.HttpStatusCodeAttribute)}")
  print(f"Error code: {reply.error()}") 
  print(f"ErrorString: {reply.errorString()}")
  #print(reply.readAll())
  return reply


failingURL = qt.QUrl("https://discourse.slicer.org") # no exception for zscaler, ssl handshake fails
passingURL = qt.QUrl("https://slicer-packages.kitware.com") # exception in place for zscaler, data returned

The passingURL returns response code 200 and the html content of the page. The failingURL returns

HTTP response code: None
Error code: 6
ErrorString: SSL handshake failed

So, I think this method seems like it will work for checking if a sufficient set of certificates has been added to Slicer.crt. I tried pasting what I got out of Chrome into the bottom of Slicer.crt, restarted Slicer, and ran this test, with no change in result. However, I don’t know if that is because I grabbed the wrong certificate (maybe I need all the certificates in the hierarchy? or just the top one? or just the bottom one?) or if it is because the certificate I tried to add was ignored because it was missing metadata or otherwise was not the proper format. For testing, do I need to restart Slicer every time I modify Slicer.crt, or is that dynamically checked each time? If there is a malformed certificate added, will any errors show up (I don’t see anything obvious in the error log, but I might also have missed something)?

@jamesobutler, to get the certificate from Chrome, I click this button next to the url in the browser window image, then click “Connection is secure >”, then “Certificate is valid”, which pops up a window, then click on the “Details” tab, and there, there is an “Export” button in the lower right. Just discovered as I was writing this out for you that in the “Export” window, it is possible to change the file type from single certificate to certificate chain:
image
If I do this, for the slicer discourse site, I get a chain of four certificates, only the begin/end certificate blocks, no other metadata. I’ll try pasting this at the end of the Slicer.crt file and see if that works. It did not, same error on the failingURL.

29

It turns out I wasn’t fully turning Zscaler off, just one component of it. Once fully turned off, the Zscaler certificates no longer show up in the browser and the failingURL above no longer gives an error in Slicer.

It remains true that adding the 4 certificates (just the begin/end blocks) exported from Chrome to the Slicer.crt file does not allow access from Slicer whenever Zscaler is on.

So, it seems like turning Zscaler off when using the Extension manager is likely a functional workaround in general. In pursuit of a non-workaround solution, I’m not sure what to try next…

1 Like
30

Yes, I do not individually turn off Zscaler component, I completely sign off and exit the application.

1 Like
31

At the time it was implemented, it was the simplest way to have certificates across all platform. The Slicer.crt is generated from the Mozilla certificates similarly to the ones distributed through python-certifi. They are also automatically maintained up-to-date using the update-slicer-certificate-bundle.yml GitHub workflow.

We could revisit this so that Python, CURL and Qt all use certificates when from the system store on platform supporting it.

1 Like