rustup nightly install failing

[davidroeca]

Just getting back into rust and installed rustup. Tried to install the nightly build and encountered the following error:

$ rustup install nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: downloading component 'rustc'
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: installing component 'rustc'
info: rolling back changes
error: failed to extract package
info: caused by: failed to unpack `rustc-nightly-x86_64-unknown-linux-gnu/rustc/share/man/man1/rustdoc.1` into `/home/david/.rustup/tmp/wnj79prt3m8vg6f2_dir/rustc/share/man/man1/rustdoc.1`
info: caused by: No such file or directory (os error 2)

Not sure if this is a nightly issue, a rustup issue, or my own lack of experience with rustup. Currently running Linux Mint 18

[messense]

Same issue on macOS 11.12

[est31]

I can confirm on Ubuntu 17.04. rustup update is broken as well. @davidroeca this is a very recent issue.

[gyscos]

Here is what I get after installing the current rustup from git (to get debug info and backtraces):

% ~/.cargo/bin/rustup update nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: downloading component 'rustc'
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: downloading component 'rust-analysis'
info: downloading component 'rls'
info: downloading component 'rust-src'
info: downloading component 'rust-std' for 'asmjs-unknown-emscripten'
info: downloading component 'rust-std' for 'wasm32-unknown-emscripten'
info: installing component 'rustc'
info: rolling back changes
error: failed to extract package
info: caused by: failed to unpack `rustc-nightly-x86_64-unknown-linux-gnu/rustc/share/man/man1/rustdoc.1` into `/home/gyscos/.rustup/tmp/_mffp8148_7wf8w9_dir/rustc/share/man/man1/rustdoc.1`
info: caused by: No such file or directory (os error 2)
info: backtrace:

stack backtrace:
   0:     0x55f2010a2894 - backtrace::backtrace::libunwind::trace
                        at /home/gyscos/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.0/src/backtrace/libunwind.rs:53
                         - backtrace::backtrace::trace<closure>
                        at /home/gyscos/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.0/src/backtrace/mod.rs:42
   1:     0x55f2010a372f - backtrace::capture::{{impl}}::new
                        at /home/gyscos/fetched/rustup.rs/target/debug/build/backtrace-6ddf4a4263d5dc15/out/capture.rs:79
   2:     0x55f20109ae3c - error_chain::make_backtrace
                        at /home/gyscos/.cargo/registry/src/github.com-1ecc6299db9ec823/error-chain-0.7.1/src/lib.rs:383
   3:     0x55f200b8032c - core::ops::FnOnce::call_once<fn() -> core::option::Option<alloc::arc::Arc<backtrace::capture::Backtrace>>,()>
                        at /checkout/src/libcore/ops.rs:2626
   4:     0x55f200b5926c - core::option::{{impl}}::or_else<alloc::arc::Arc<backtrace::capture::Backtrace>,fn() -> core::option::Option<alloc::arc::Arc<backtrace::capture::Backtrace>>>
                        at /checkout/src/libcore/option.rs:640
   5:     0x55f200b44c6a - error_chain::{{impl}}::new<rustup_dist::errors::Error>
                        at /home/gyscos/.cargo/registry/src/github.com-1ecc6299db9ec823/error-chain-0.7.1/src/lib.rs:437
   6:     0x55f200bf8a5b - rustup_dist::errors::{{impl}}::chain_err::{{closure}}<(),std::io::error::Error,closure,rustup_dist::errors::ErrorKind>
                        at /home/gyscos/fetched/rustup.rs/<error_chain_processed macros>:117
   7:     0x55f200b74c46 - core::result::{{impl}}::map_err<(),std::io::error::Error,rustup_dist::errors::Error,closure>
                        at /checkout/src/libcore/result.rs:486
   8:     0x55f200bf79d1 - rustup_dist::errors::{{impl}}::chain_err<(),std::io::error::Error,closure,rustup_dist::errors::ErrorKind>
                        at /home/gyscos/fetched/rustup.rs/<error_chain_processed macros>:115
   9:     0x55f200bdacde - rustup_dist::component::package::unpack_without_first_dir<flate2::gz::DecoderReader<std::fs::File>>
                        at src/rustup-dist/src/component/package.rs:202
  10:     0x55f200bd9f7b - rustup_dist::component::package::{{impl}}::new<flate2::gz::DecoderReader<std::fs::File>>
                        at src/rustup-dist/src/component/package.rs:183
  11:     0x55f200bdb4f9 - rustup_dist::component::package::{{impl}}::new<std::fs::File>
                        at src/rustup-dist/src/component/package.rs:232
  12:     0x55f200bdba4c - rustup_dist::component::package::{{impl}}::new_file
                        at src/rustup-dist/src/component/package.rs:236
  13:     0x55f200be427b - rustup_dist::manifestation::{{impl}}::update
                        at src/rustup-dist/src/manifestation.rs:170
  14:     0x55f200bcc817 - rustup_dist::dist::update_from_dist_
                        at src/rustup-dist/src/dist.rs:656
  15:     0x55f200bcbcce - rustup_dist::dist::update_from_dist
                        at src/rustup-dist/src/dist.rs:619
  16:     0x55f200b1e26e - rustup::install::{{impl}}::run
                        at src/rustup/install.rs:50
  17:     0x55f200b09851 - rustup::toolchain::{{impl}}::install
                        at src/rustup/toolchain.rs:113
  18:     0x55f200b0aa6f - rustup::toolchain::{{impl}}::install_from_dist_inner
                        at src/rustup/toolchain.rs:175
  19:     0x55f200b0a5b5 - rustup::toolchain::{{impl}}::install_from_dist
                        at src/rustup/toolchain.rs:170
  20:     0x55f2009f7a2d - rustup_init::rustup_mode::update
                        at src/rustup-cli/rustup_mode.rs:488
  21:     0x55f2009ec639 - rustup_init::rustup_mode::main
                        at src/rustup-cli/rustup_mode.rs:33
  22:     0x55f200a15dd8 - rustup_init::run_multirust
                        at src/rustup-cli/main.rs:82
  23:     0x55f200a15b86 - rustup_init::main
                        at src/rustup-cli/main.rs:57
  24:     0x55f2010fc15a - panic_unwind::__rust_maybe_catch_panic
                        at /checkout/src/libpanic_unwind/lib.rs:98
  25:     0x55f2010f59b1 - std::panicking::try<(),fn()>
                        at /checkout/src/libstd/panicking.rs:433
                         - std::panic::catch_unwind<fn(),()>
                        at /checkout/src/libstd/panic.rs:361
                         - std::rt::lang_start
                        at /checkout/src/libstd/rt.rs:57
  26:     0x55f200a1a8a2 - main
  27:     0x7f5759081510 - __libc_start_main
  28:     0x55f200971539 - _start
  29:                0x0 - <unknown>

[TimNN]

strace output:

info: installing component 'rustc'
open("/home/logic/.rustup/tmp/hmzwj4cotkanpzvj_file", O_RDONLY|O_CLOEXEC) = 3
mkdir("/home/logic/.rustup/tmp/3v47lhv9_ptrqrvj_dir", 0777) = 0
open("/home/logic/.rustup/tmp/3v47lhv9_ptrqrvj_dir/rustc/share/man/man1/rustdoc.1", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = -1 ENOENT (No such file or directory)
open("/home/logic/.rustup/tmp/3v47lhv9_ptrqrvj_dir", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3

[TimNN]

Ok, the problem seems to be that the tar.gz generation changed and no longer includes directory entries, so the tar.gz extractor used by rustup no longer creates any directories.

[jonhoo]

Can confirm on Arch Linux as well.

[alexcrichton]

@ranma42 this is likely related to rust-lang/rust#41600, notably this range of changes, mind taking a look?

[brson]

Thanks for investigating @TimNN.

I might guess it was this PR updating rust-installer that caused the change on Rust's side.

[brson]

Damn you @alexcrichton!

[TimNN]

This causes the problem, I think rust-lang/rust-installer@4f99485...4cf7397#diff-534b12d64247754d4938410a9ba07bd3R274

find "$CFG_INPUT" \( -type d -empty \) -or \( -not -type d \) \
    | rev | sort | rev | tar -cf "$CFG_OUTPUT.tar" -T -
need_ok "failed to tar"

Not a find expert here, but I think that means that only empty directories are included in the output .tar (and files, of course).

[TimNN]

Rustup does the unpacking of tar files here: https://github.com/rust-lang-nursery/rustup.rs/blob/master/src/rustup-dist/src/component/package.rs#L202, however unpack requires all intermediary directories to have been created: https://docs.rs/tar/0.4.11/tar/struct.Entry.html#method.unpack

[brson]

Is it obvious to anyone how to change the tar command to include the directories again? I think we could just remove all of "( -type d -empty ) -or ( -not -type d ) " but I'm not sure.

Safe thing would be to just revert rust-installer.

@alexcrichton What time does the next nightly start building?

[alexcrichton]

@brson needs to be past @bors by 0:00 UTC which gives us ~19 hours from this comment

[jonhoo]

@brson I believe the find options can be removed, and in fact that point we could probably just use tar directly instead of using find at all..