Breaking Linux downloads

Support
,
1

Unable to download 3D Slicer for Linux.

$ wget -c https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download
Saving 'download'
Just got 32571392 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 32571392.
Saving 'download'
Just got 9109504 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 9109504.
Saving 'download'
Just got 14090240 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 14090240.
Saving 'download'
Just got 6422528 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 6422528.

The connection is unexpectedly terminated with 200. And resume is not supported. Looks like a bug on server side.

2

These are working for me:

For the preview:

wget "http://download.slicer.org/download?os=linux&stability=any&offset=0" -O /tmp/slicer.tar.gz

For 5.8.1:

wget "https://download.slicer.org/bitstream/67c51fc129825655577cfee9" -O /tmp/slicer-stable.tar.gz

Maybe you have a proxy or other networking issue?

(edited to replace -o with -O)

3

I moved to another place with another ISP. There is a better speed, about 400KB/s, but the connection breaks too. It looks like I doesn’t reach 50% when it restarts.

$ wget -c https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download
Saving 'download'
Just got 169148416 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 169148416.
Saving 'download'
Just got 31391744 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 31391744.
Saving 'download'
Just got 153157632 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 153157632.
Saving 'download'
Just got 11272192 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 11272192.
Saving 'download'
Just got 10260480 of 402316524 bytes (https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download)
HTTP response 200 OK [https://slicer-packages.kitware.com/api/v1/item/67c51fc129825655577cfee9/download]
Unexpected body length 10260480.

                          [Files: 0  Bytes: 0  [0 B/s] Redirects: 0  Todo: 1  Errors: 0                                                          ]

I am not sure what is the server configuration, but judging that https://github.com/Slicer/slicer_download is Python app, it could be that Nginx breaks the connection under heavy load.

Now I was able to download slicer and the speed is much higher (3MB/s) than few hour ago when the download was broken.

I

4

So to improve the download experience, the Python + uWSGI + Nginx could probably be replaced with Go server that supports resuming downloads out of the box. uWSGI seems dead ( The uWSGI project — uWSGI 2.0 documentation ) so probably going with Go http.FileServe() is the easiest quick win.

EDIT: code to start static-server: an HTTP server in Go for static content - Eli Bendersky's website

5

Slicer uses girder as storage backend for application packages. Unfortunately, girder has a bug: HTTP range requests are implemented incorrectly It would be OK if the server did not support it, but instead the server claims it can do it, but actually ignores the requested range parameters. The issue is described in detail here:

I’ve been downloading Slicer on various computers over various internet connections and usually the speed is between 3 MB/sec and 100 MB/sec. The entire package is downloaded within one minute, there are no interruptions, no need for resumes, so the HTTP range request bug has no impact. If the download speed for some reason is unusually slow (e.g., 0.3MB/sec), then the connection may be interrupted and your download client may then try to use “resume” feature (HTTP range request), which will result in an invalid downloaded file or other failure.

The problem will get higher priority if we get more complaints or we can get significant funding for improving the hosting infrastructure. But for now the approach will remain “watch and wait”.

1 Like