Consider providing zstd tarballs on static.r-l.o

[dtolnay]

To compensate for #89.

Decompression is probably still faster with .gz but if we care about that zstd-compression is probably the better option.

I benchmarked rustc + rust-std + cargo + clippy + rustfmt + llvm-tools to help choose a compression level:

compression	size (MB)	extraction time (sec)
none	649	N/A
gz	215	3.6
xz	127	8.4
zstd -1	232	1.2
zstd -2	217
zstd -3	207	1.2
zstd -4	203
zstd -5	198
zstd -6	193
zstd -7	190
zstd -8	188
zstd -9	184
zstd -10	182	1.25
zstd -11	182
zstd -12	181
zstd -13	181
zstd -14	181
zstd -15	180
zstd -16	174
zstd -17	166
zstd -18	159
zstd -19	158	1.4

[dtolnay]

At level 19, decompressing in 1.4 seconds is significantly faster than 8.4 seconds for xz. Users will notice this.

The tradeoff is that this loses 35% of the bandwidth benefit of #89.

The user must also be able to download 31 MB in 7 seconds (35 Mbps; 4.4 MiB/s) in order to see any benefit. For comparison, I get 90+ MiB/s from static.rust-lang.org so zstd would definitely be a speedup for me. Perhaps rustup could choose between xz and zstd based on the observed speed. I imagine rust-lang/rustup#731 would also be relevant to choosing optimally.

[dtolnay]

Relevant discussion on the rustup side:

• Prefer faster compression methods under good network rustup#1858
• Add support for .zst for component tarballs in channels rustup#2488
• Add zstd support rustup#2676

[criemen]

Is there any update on this? I'm using rules_rust in CI (github actions), so I'm downloading rust a lot, and we're really suffering those decompression times. It'd be great if we could get official zst tarballs, at least for releases!