Clang requirement when building for WebAssembly, `cc-rs` through `zstd`, fails

[chrisvander]

Describe the bug

This commit introduces a non-optional dependency on zstd, which in turn relies on zstd-safe and zstd-sys, which uses C++ compilation through the cc crate and requires clang. This breaks our WebAssembly build with no option to opt-out to the clang call, even if we're not using compression.

I'll also note, with a native build on macOS with Clang 17.0.0, it fails with:

warning: zstd-sys@2.0.15+zstd.1.5.7: error: unable to create target: 'No available targets are compatible with triple "wasm32-unknown-unknown"'

We began using main to fix a different issue, namely to avoid the recursive crate, since Rust's WASM target only supports that compilation with the GNU toolchain since it, too, uses some C++. We don't expect that the crate is entirely WASM-safe. We planned to switch back to major releases with version 49.0.0.

Note that the official Rust Docker images do not include the tooling to build with Clang, either.

Build failure looks like this:

11.20 warning: zstd-sys@2.0.15+zstd.1.5.7: Compiler family detection failed due to error: ToolNotFound: failed to find tool "clang": No such file or directory (os error 2)
11.20 warning: zstd-sys@2.0.15+zstd.1.5.7: Compiler family detection failed due to error: ToolNotFound: failed to find tool "clang": No such file or directory (os error 2)
11.20 warning: zstd-sys@2.0.15+zstd.1.5.7: Compiler family detection failed due to error: ToolNotFound: failed to find tool "clang": No such file or directory (os error 2)
11.20 warning: zstd-sys@2.0.15+zstd.1.5.7: Compiler family detection failed due to error: ToolNotFound: failed to find tool "clang": No such file or directory (os error 2)
11.20 error: failed to run custom build command for `zstd-sys v2.0.15+zstd.1.5.7`
11.20 note: To improve backtraces for build dependencies, set the CARGO_PROFILE_RELEASE_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.
11.20
11.20 Caused by:
11.20   process didn't exit successfully: `/app/target/release/build/zstd-sys-f761fb7845587b14/build-script-build` (exit status: 1)
11.20   --- stdout
11.20   cargo:rerun-if-env-changed=ZSTD_SYS_USE_PKG_CONFIG
11.20   cargo:rustc-cfg=feature="std"
11.20   cargo:rerun-if-changed=wasm-shim/stdlib.h
11.20   cargo:rerun-if-changed=wasm-shim/string.h
11.20   OUT_DIR = Some(/app/target/wasm32-unknown-unknown/release/build/zstd-sys-8eb71b94f934d217/out)
11.20   OPT_LEVEL = Some(3)
11.20   TARGET = Some(wasm32-unknown-unknown)
11.20   HOST = Some(aarch64-unknown-linux-gnu)
11.20   cargo:rerun-if-env-changed=CC_wasm32-unknown-unknown
11.20   CC_wasm32-unknown-unknown = None
11.20   cargo:rerun-if-env-changed=CC_wasm32_unknown_unknown
11.20   CC_wasm32_unknown_unknown = None
11.20   cargo:rerun-if-env-changed=TARGET_CC
11.20   TARGET_CC = None
11.20   cargo:rerun-if-env-changed=CC
11.20   CC = None
11.20   cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
11.20   cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "clang": No such file or directory (os error 2)

To Reproduce

Run a WebAssembly build, with Datafusion as such:

datafusion = { version = "48.0.0", features = ["backtrace", "datetime_expressions", "nested_expressions", "string_expressions"], default-features = false }

and a patch

datafusion = { version = "48.0.0", git = "https://github.com/apache/datafusion.git", rev = "350c61b23a2732ebd8e73cf08fe168cf6d73fb5b" }

Expected behavior

The recommended solution would be to put zstd behind a feature flag. We're careful about what dependencies we're using and, for the future, if there are any non-pure Rust dependencies that may make WebAssembly or Docker builds a problem, keep them behind feature flags.

Additional context

No response