# Slicer fails to download custom sample data

**URL:** <https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561>\
**Category:** Development\
**Tags:** python, extensions\
**Created:** [March 24, 2023, 5:43pm UTC](https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561 "2023-03-24T17:43:14Z")\
**Posts on this page:** 6\
**Page:** 2

<div class="post-metadata">

**Author:** ![dzenanz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/dzenanz/32/1992_2.png) [@dzenanz](https://discourse.slicer.org/u/dzenanz)\
**Post date:** [April 7, 2023, 1:44pm UTC](https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561/21 "2023-04-07T13:44:04Z")

</div>

I have narrowed the problem down. This script works:

```py
import sys
import traceback
import urllib.request
import requests

def processEvents():
    try:
        import slicer
        slicer.app.processEvents()
    except (AttributeError, ImportError):
        pass

    
def display(text, file=sys.stdout):
    processEvents()
    print(text, flush=True, file=file)

    
def error(text):
    display(text, file=sys.stderr)

for url, expected_success in [
    ("https://data.kitware.com", True),
    ("https://www.httpvshttps.com/", True),
    ("https://slicer.org/", True),
    ("https://expired.badssl.com/", False),
    ("https://github.com/", True),
]:
    display("-" * 8)
    display(f"Checking {url}")
    goodCert = r"C:\Program Files\Python39\Lib\site-packages\certifi\cacert.pem"
    badCert = requests.utils.DEFAULT_CA_BUNDLE_PATH # C:\Users\Dzenan\AppData\Local\.certifi
    try:
        with urllib.request.urlopen(url, cafile=goodCert) as response:
           data = response.read()
           html = data.decode('utf8')
           assert "<head>" in html[:600]
        if expected_success:
            display(f"Checking {url} - OK")
        else:
            error(f"Checking {url} - FAILED - should have raised an exception")
    except Exception as exc:
        if expected_success:
            error(f"Checking {url} - FAILED - unexpected exception")
            traceback.print_exc()
        else:
            display(f"Checking {url} - OK [Expected {exc}]")

```

I tried copying `goodCert` into `badCert`’s place. But when this script is run, the `goodCert` gets turned into `badCert` by extending it by a bunch more entries. The file’s size is increased from 253KB to 425KB.

Does anyone have an idea what to try next?

---

<div class="post-metadata">

**Author:** ![dzenanz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/dzenanz/32/1992_2.png) [@dzenanz](https://discourse.slicer.org/u/dzenanz)\
**Post date:** [April 7, 2023, 5:00pm UTC](https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561/22 "2023-04-07T17:00:23Z")

</div>

I have now narrowed it down to a single certificate.

```plaintext
-----BEGIN CERTIFICATE-----
MIIEZTCCA02gAwIBAgIQQAF1BIMUpMghjISpDBbN3zANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTIwMTAwNzE5MjE0MFoXDTIxMDkyOTE5MjE0MFow
MjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxCzAJBgNVBAMT
AlIzMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuwIVKMz2oJTTDxLs
jVWSw/iC8ZmmekKIp10mqrUrucVMsa+Oa/l1yKPXD0eUFFU1V4yeqKI5GfWCPEKp
Tm71O8Mu243AsFzzWTjn7c9p8FoLG77AlCQlh/o3cbMT5xys4Zvv2+Q7RVJFlqnB
U840yFLuta7tj95gcOKlVKu2bQ6XpUA0ayvTvGbrZjR8+muLj1cpmfgwF126cm/7
gcWt0oZYPRfH5wm78Sv3htzB2nFd1EbjzK0lwYi8YGd1ZrPxGPeiXOZT/zqItkel
/xMY6pgJdz+dU/nPAeX1pnAXFK9jpP+Zs5Od3FOnBv5IhR2haa4ldbsTzFID9e1R
oYvbFQIDAQABo4IBaDCCAWQwEgYDVR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8E
BAMCAYYwSwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5p
ZGVudHJ1c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTE
p7Gkeyxx+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEE
AYLfEwEBATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2Vu
Y3J5cHQub3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0
LmNvbS9EU1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYf
r52LFMLGMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjANBgkqhkiG9w0B
AQsFAAOCAQEA2UzgyfWEiDcx27sT4rP8i2tiEmxYt0l+PAK3qB8oYevO4C5z70kH
ejWEHx2taPDY/laBL21/WKZuNTYQHHPD5b1tXgHXbnL7KqC401dk5VvCadTQsvd8
S8MXjohyc9z9/G2948kLjmE6Flh9dDYrVYA9x2O+hEPGOaEOa1eePynBgPayvUfL
qjBstzLhWVQLGAkXXmNs+5ZnPBxzDJOLxhF2JIbeQAcH5H0tZrUlo5ZYyOqA7s9p
O5b85o3AM/OJ+CktFBQtfvBhcJVd9wvlwPsk+uyOy2HI7mNxKKgsBTt375teA2Tw
UdHkhVNcsAKX1H7GNNLOEADksd86wuoXvg==
-----END CERTIFICATE-----

```

human readable variant:

```plaintext
Certificate[129]:
Owner: CN=R3, O=Let's Encrypt, C=US
Issuer: CN=DST Root CA X3, O=Digital Signature Trust Co.
Serial number: 400175048314a4c8218c84a90c16cddf
Valid from: Wed Oct 07 15:21:40 EDT 2020 until: Wed Sep 29 15:21:40 EDT 2021
Certificate fingerprints:
SHA1: 48:50:4E:97:4C:0D:AC:5B:5C:D4:76:C8:20:22:74:B2:4C:8C:71:72
SHA256: 73:0C:1B:DC:D8:5F:57:CE:5D:C0:BB:A7:33:E5:F1:BA:5A:92:5B:2A:77:1D:64:0A:26:F7:A4:54:22:4D:AD:3B
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions:

#1: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
AuthorityInfoAccess [
  [
   accessMethod: caIssuers
   accessLocation: URIName: http://apps.identrust.com/roots/dstrootcax3.p7c
]
]

#2: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: C4 A7 B1 A4 7B 2C 71 FA DB E1 4B 90 75 FF C4 15 .....,q...K.u...
0010: 60 85 89 10 `...
]
]

#3: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
  CA:true
  PathLen:0
]

#4: ObjectId: 2.5.29.31 Criticality=false
CRLDistributionPoints [
  [DistributionPoint:
     [URIName: http://crl.identrust.com/DSTROOTCAX3CRL.crl]
]]

#5: ObjectId: 2.5.29.32 Criticality=false
CertificatePolicies [
  [CertificatePolicyId: [2.23.140.1.2.1]
[] ]
  [CertificatePolicyId: [1.3.6.1.4.1.44947.1.1.1]
[PolicyQualifierInfo: [
  qualifierID: 1.3.6.1.5.5.7.2.1
  qualifier: 0000: 16 22 68 74 74 70 3A 2F 2F 63 70 73 2E 72 6F 6F ."http://cps.roo
0010: 74 2D 78 31 2E 6C 65 74 73 65 6E 63 72 79 70 74 t-x1.letsencrypt
0020: 2E 6F 72 67 .org

]] ]
]

#6: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
  clientAuth
]

#7: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_CertSign
  Crl_Sign
]

#8: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 14 2E B3 17 B7 58 56 CB AE 50 09 40 E6 1F AF 9D .....XV..P.@....
0010: 8B 14 C2 C6 ....
]
]

```

If this certificate is not present in cacert.pem, the script works as it should. If this certificate is present, all websites have “certificate has expired” problem within Python.

I tried deleting it from the local computer, but it reappeared.

 ![Screenshot 2023-04-07 12.49.16](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/f/7/f7319386a92a10d3569b1dc1a38b28f0208aec38.jpeg)

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [April 7, 2023, 5:14pm UTC](https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561/23 "2023-04-07T17:14:06Z")

</div>

The `DST Root CA X3` certificate expired \[1\] in September 2021.

Consider updating the version of `certifi` associated with your Python install. See [https://pypi.org/project/certifi/](https://pypi.org/project/certifi/)

> But when this script is run, the `goodCert` gets turned into `badCert` by extending it by a bunch more entries.

Involved functions are the following:

- [https://docs.python.org/3/library/urllib.request.html#urllib.request.urlopen](https://docs.python.org/3/library/urllib.request.html#urllib.request.urlopen)
- [https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load\_verify\_locations](https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_verify_locations)

* * *

1. [DST Root CA X3 Expiration (September 2021) - Let's Encrypt](https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/)

---

<div class="post-metadata">

**Author:** ![dzenanz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/dzenanz/32/1992_2.png) [@dzenanz](https://discourse.slicer.org/u/dzenanz)\
**Post date:** [April 7, 2023, 8:13pm UTC](https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561/24 "2023-04-07T20:13:35Z")

</div>

JC and I had a debugging session on my computer. We didn’t resolve the problem. But JC found this:

> Starting with Python 2.7.9 and 3.2, the function “ssl.create\_default\_context()” automatically loads system certificates. This explains why the “Slicer.crt” we provide (through the SSL\_CERT\_FILE env. variable expected by openssl) is being ignored

---

<div class="post-metadata">

**Author:** ![dzenanz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/dzenanz/32/1992_2.png) [@dzenanz](https://discourse.slicer.org/u/dzenanz)\
**Post date:** [April 7, 2023, 8:22pm UTC](https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561/25 "2023-04-07T20:22:20Z")

</div>

I also figured out why the certificate comes back after I delete it:  
 ![Screenshot 2023-04-07 15.49.39](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/a/6/a6330fab6af558ce741eb1b8f21a6ec54b992df9.png)  
And in text form for easier searching:

> Deleting system root certificates might prevent some Windows components from working properly. The list of system critical root certificates can be reviewed at [Microsoft Support](https://support.microsoft.com/?id=293781). If Update Root Certificates is installed, any deleted third-party root certificates will be restored automatically, but the system root certificates will not. Do you want to delete the selected certificate(s)?

---

<div class="post-metadata">

**Author:** ![dzenanz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/dzenanz/32/1992_2.png) [@dzenanz](https://discourse.slicer.org/u/dzenanz)\
**Post date:** [April 7, 2023, 8:50pm UTC](https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561/26 "2023-04-07T20:50:29Z")

</div>

Downloading and installing certificates from [this website](https://www.stephenwagner.com/2021/09/30/sophos-dst-root-ca-x3-expiration-problems-fix/) solved the problem for me. Here are the links:

> Root CA Certificates (PEM format):
> 
> - [ISRG Root X1](https://letsencrypt.org/certs/isrgrootx1.pem) (Or [ISRG Root X1 DER Format](https://letsencrypt.org/certs/isrgrootx1.der))
> - [ISRG Root X2](https://letsencrypt.org/certs/isrg-root-x2.pem) (Or [ISRG Root X2 DER Format](https://letsencrypt.org/certs/isrg-root-x2.der))
> 
> Intermediate Certificate (PEM format):
> 
> - [Let’s Encrypt R3](https://letsencrypt.org/certs/lets-encrypt-r3.pem) (Or [Let’s Encrypt R3 DER Format](https://letsencrypt.org/certs/lets-encrypt-r3.der))

Thank you JC, Andras and @jake.stookey for the help.

[Previous page](https://discourse.slicer.org/t/slicer-fails-to-download-custom-sample-data/28561.md?page=1)