Support resume of partial downloads

[johalun]

First of all, thanks for a good job. Rustup makes things very convinient and works well while on stable and fast internet.

However, I'm in China on business trip and trying to install Rust using Rustup from here. Other than painfully slow downloads, download is often interrupted and I have to restart from scratch.

Having a local mirror here in China would be awesome for all the people here but that might be tricky so at least, it would be nice with support for partial downloads and the ability to resume broken download.

Thank you

[sam-falvo]

I second this issue. I'm in San Francisco Bay Area, on 12Mbps (ostensibly!) business-class cable modem connection. The actual connection often drops to single-digit bits per second (!!!) for extended periods of time due to high packet loss (>48%) deep within Comcast's fabric. In addition, as of my posting this message on December 29, 2016, it looks like AWS is also having exceptionally high packet losses as well. I'd include a traceroute here if it weren't so transient, and if I had the knowledge that someone on the rustup team had the power to address the issue.

Features desired includes one or more of the following:

• An option to turn off the timeout. Alternatively, an option to set my own timeout. I can try to add this feature myself (I think it's just a curl option) and submit a PR, but I wanted to document the issue here for reference.
• An option to enable or disable auto-resume functionality. I'm a little less knowledgeable about how to make this work. curl has options to support resume, IIRC, but I think logic in the initial shell script will need to be changed to preserve downloaded files. I just skimmed it, but in skimming, I haven't seen where to make this logic change.

[johalun]

I'm not sure but resume might have to be supported by the server as well. If this is not easily done, maybe it could at least cache the downloaded files so that we don't have to re-download all files when it fails on cargo download in the end..

[sam-falvo]

I tried to spend some time with the current software, and while I could find the one curl command that downloads the next install stage script (the one that actually downloads and installs the components), finding out where those install scripts are located has been something of a question mark for me. Maybe I'm not seeing the obvious, but when I visit https://github.com/rust-lang-nursery, I do not see another candidate repository in which those scripts would reside.

Where else should I look?

Thanks.

[kosinix]

Seconded. Everytime rustup fails on my slow internet connection, it has to start all over again its frustrating. I could not install rust for that matter. Is there a viable workaround for this at the moment? Maybe download each components manually?

[brson]

The most important downloads to resume are the installer tarballs. The place to look to update is probably Manifestation::update. There's a phase there where all tarballs are downloaded.

The way multirust.sh did this (and the way I suggest for rustup) is to add a ~/.rustup/downloads directory that contains files named after their sha256 sum (which we know a priori). Download the files there, where they can be resumed, then move them into place.

The other place where resumption may matter, but not as much, is when downloading rustup-init.exe, which is a few MB today. This needs to be done in rustup-init.sh ala #894.

[yami]

second this. I'm in China, it is frustrating that rustup downloads everything after interruption due to network issues.

[J-F-Liu]

Should at lease cache downloaded components, here is my console log:

▶ rustup update nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
143.3 KiB / 143.3 KiB (100 %)  71.6 KiB/s ETA:   0 s                
info: downloading component 'rustc'
 46.4 MiB /  46.4 MiB (100 %)  41.6 KiB/s ETA:   0 s                       
info: downloading component 'rust-std'
 61.4 MiB /  61.4 MiB (100 %)  60.8 KiB/s ETA:   0 s                        
info: downloading component 'cargo'
  1.5 MiB /   4.4 MiB ( 34 %)  13.6 KiB/s ETA: 217 s                error: component download failed for cargo-x86_64-unknown-linux-gnu
info: caused by: could not download file from 'https://s3.amazonaws.com/rust-lang-ci/cargo-builds/6c79cf9cd24756844bda852de6bd3df29b798991/cargo-nightly-x86_64-unknown-linux-gnu.tar.gz' to '/home/junfeng/.rustup/tmp/_dib_w3x6xb11rkj_file
info: caused by: error during download
info: caused by: [28] Timeout was reached (Operation too slow. Less than 10 bytes/sec transferred the last 30 seconds)

/work/pom  master ✗                                                                                                   51m ⚑  ⍉
▶ rustup update nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
143.3 KiB / 143.3 KiB (100 %) 112.0 KiB/s ETA:   0 s                
info: downloading component 'rustc'
 46.4 MiB /  46.4 MiB (100 %)  51.2 KiB/s ETA:   0 s                       
info: downloading component 'rust-std'
 61.4 MiB /  61.4 MiB (100 %)  56.3 KiB/s ETA:   0 s                       
info: downloading component 'cargo'
  1.3 MiB /   4.4 MiB ( 30 %)  13.6 KiB/s ETA: 232 s                error: component download failed for cargo-x86_64-unknown-linux-gnu
info: caused by: could not download file from 'https://s3.amazonaws.com/rust-lang-ci/cargo-builds/6c79cf9cd24756844bda852de6bd3df29b798991/cargo-nightly-x86_64-unknown-linux-gnu.tar.gz' to '/home/junfeng/.rustup/tmp/9x4xgxcj5fcwox6z_file
info: caused by: error during download
info: caused by: [28] Timeout was reached (Operation too slow. Less than 10 bytes/sec transferred the last 30 seconds)