Skip to content

Commit 4018cc7

Browse files
committed
[SPARK-56338][INFRA][FOLLOWUP] Support MAVEN_MIRROR_URL in SBT launcher bootstrap
### What changes were proposed in this pull request? This is a follow-up to apache#55168. The SBT launcher script (`build/sbt-launch-lib.bash`) did not respect `MAVEN_MIRROR_URL`. This patch adds support for it in two places: 1. **Launcher JAR download**: `MAVEN_MIRROR_URL` is used as the default for `DEFAULT_ARTIFACT_REPOSITORY` when neither is explicitly set. 2. **SBT boot phase**: When `MAVEN_MIRROR_URL` is set, a temporary SBT repositories config is generated and passed via `-Dsbt.repository.config` so SBT resolves itself and Scala through the mirror. ### Why are the changes needed? SPARK-56338 added `MAVEN_MIRROR_URL` support for Maven (`pom.xml`) and SBT project builds (`SparkBuild.scala`), but the SBT launcher script was not covered. In environments where default Maven repositories are unreachable, the SBT launcher JAR download and boot phase still fail without manual configuration (e.g. `~/.sbt/repositories`). ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually tested by setting `MAVEN_MIRROR_URL` and building Spark with SBT from scratch (launcher JAR removed, no `~/.sbt/repositories`). ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code Closes apache#55238 from cloud-fan/spark-56338-sbt-mirror. Authored-by: Wenchen Fan <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent cf26c42 commit 4018cc7

File tree

1 file changed

+18
-3
lines changed

1 file changed

+18
-3
lines changed

build/sbt-launch-lib.bash

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,11 +39,12 @@ dlog () {
3939

4040
acquire_sbt_jar () {
4141
SBT_VERSION=`awk -F "=" '/sbt\.version/ {print $2}' ./project/build.properties`
42-
# DEFAULT_ARTIFACT_REPOSITORY env variable can be used to only fetch
43-
# artifacts from internal repos only.
42+
# Artifacts are fetched from DEFAULT_ARTIFACT_REPOSITORY if set, then
43+
# MAVEN_MIRROR_URL, and finally the default Google mirror of Maven Central.
4444
# Ex:
4545
# DEFAULT_ARTIFACT_REPOSITORY=https://artifacts.internal.com/libs-release/
46-
DEFAULT_ARTIFACT_REPOSITORY=${DEFAULT_ARTIFACT_REPOSITORY:-https://maven-central.storage-download.googleapis.com/maven2/}
46+
local default_repo=${MAVEN_MIRROR_URL:-https://maven-central.storage-download.googleapis.com/maven2/}
47+
DEFAULT_ARTIFACT_REPOSITORY=${DEFAULT_ARTIFACT_REPOSITORY:-$default_repo}
4748
URL1=${DEFAULT_ARTIFACT_REPOSITORY%/}/org/scala-sbt/sbt-launch/${SBT_VERSION}/sbt-launch-${SBT_VERSION}.jar
4849
JAR=build/sbt-launch-${SBT_VERSION}.jar
4950

@@ -182,6 +183,20 @@ run() {
182183
set -- "${residual_args[@]}"
183184
argumentCount=$#
184185

186+
# If MAVEN_MIRROR_URL is set, generate a repositories config so the SBT launcher
187+
# resolves SBT and Scala through the mirror during the boot phase.
188+
# Skip if the user already configured -Dsbt.repository.config.
189+
if [[ -n "$MAVEN_MIRROR_URL" && ! "$SBT_OPTS" =~ sbt\.repository\.config ]]; then
190+
local sbt_repo_config="$(mktemp /tmp/sbt-repositories.XXXXXX)"
191+
trap "rm -f '$sbt_repo_config'" EXIT
192+
cat > "$sbt_repo_config" <<EOF
193+
[repositories]
194+
local
195+
maven-mirror: ${MAVEN_MIRROR_URL}
196+
EOF
197+
addJava "-Dsbt.repository.config=$sbt_repo_config"
198+
fi
199+
185200
# run sbt
186201
execRunner "$java_cmd" \
187202
$(get_mem_opts $sbt_mem) \

0 commit comments

Comments
 (0)