Skip to content

[SPARK-55710][BUILD] Use Google Mirror of Maven Central for SBT bootstrap#54508

Closed
yaooqinn wants to merge 1 commit intoapache:masterfrom
yaooqinn:SPARK-55710
Closed

[SPARK-55710][BUILD] Use Google Mirror of Maven Central for SBT bootstrap#54508
yaooqinn wants to merge 1 commit intoapache:masterfrom
yaooqinn:SPARK-55710

Conversation

@yaooqinn
Copy link
Copy Markdown
Member

@yaooqinn yaooqinn commented Feb 26, 2026

What changes were proposed in this pull request?

Change the default SBT bootstrap repository from Maven Central (repo1.maven.org) to Google's Maven Central mirror (maven-central.storage-download.googleapis.com) for downloading the sbt-launch jar.

Why are the changes needed?

The rest of the SBT build already uses the Google mirror as the primary resolver, as project/SparkBuild.scala. The bootstrap script (build/sbt-launch-lib.bash) was the only place still defaulting to Maven Central. This makes it consistent and avoids flaky Maven Central connections during the initial sbt-launch jar download.

The DEFAULT_ARTIFACT_REPOSITORY environment variable can still be used to override this.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manually tested by removing the cached sbt-launch-1.12.4.jar and verifying the bootstrap successfully downloads from the Google mirror with matching checksum.

Was this patch authored or co-authored using generative AI tooling?

Yes, GitHub Copilot CLI was used.

…trap

Change the default SBT bootstrap repository from Maven Central (repo1.maven.org) to Google's Maven Central mirror (maven-central.storage-download.googleapis.com) for downloading the sbt-launch jar.

The rest of the SBT build already uses the Google mirror as the primary resolver (configured in project/repositories and project/SparkBuild.scala). This makes the bootstrap consistent and avoids flaky Maven Central connections.

The DEFAULT_ARTIFACT_REPOSITORY environment variable can still override this.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sarutak
Copy link
Copy Markdown
Member

sarutak commented Feb 26, 2026

@yaooqinn This change itself looks good to me. Leave some comments.

The rest of the SBT build already uses the Google mirror as the primary resolver, as configured in project/repositories and project/SparkBuild.scala.

Is project/repositories a typo for something?

I think it's OK to do in a separate PR but should we change the following code as well?
https://github.com/apache/spark/blob/master/common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala#L183

@yaooqinn
Copy link
Copy Markdown
Member Author

I revised the PR desc, thank you @sarutak

@yaooqinn
Copy link
Copy Markdown
Member Author

I think it's OK to do in a separate PR but should we change the following code as well?
https://github.com/apache/spark/blob/master/common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala#L183

This could be a user-facing change, I'm OK for it w/ a separate PR

Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

@dongjoon-hyun
Copy link
Copy Markdown
Member

Thank you, @yaooqinn , @sarutak , @LuciferYang .
Merged to master for Apache Spark 4.2.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants