[SPARK-56025][INFRA] Install `remotes` R package for `spark-test-image/(lint|docs|sparkr)/Dockerfile` by dongjoon-hyun · Pull Request #54853 · apache/spark

dongjoon-hyun · 2026-03-17T06:34:24Z

What changes were proposed in this pull request?

This PR aims to recover the CI by installing remotes R package for the following Dockerfiles.

dev/spark-test-image/lint/Dockerfile
dev/spark-test-image/docs/Dockerfile
dev/spark-test-image/sparkr/Dockerfile

Why are the changes needed?

Currently, master branch is broken due to the new devtools 2.5.0 release. This PR aims to installation remotes package explicitly because we cannot pin devtools itself.

https://github.com/r-lib/devtools/releases/tag/v2.5.0 (2026-03-16)

https://github.com/apache/spark/actions/runs/23180660599/job/67352803326

801.6 Error in devtools::install_version("pkgdown", version = "2.0.1", repos = "https://cloud.r-project.org/") : 
801.6   The package "remotes" is required.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Recover the CIs.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-opus-4-6)

…ockerfile

dongjoon-hyun · 2026-03-17T06:47:42Z

dev/spark-test-image/lint/Dockerfile

 LABEL org.opencontainers.image.version=""

-ENV FULL_REFRESH_DATE=20260210
+ENV FULL_REFRESH_DATE=20260316


This is bumped in order to make it sure to rebuild all freshly.

sarutak · 2026-03-17T06:54:12Z

@dongjoon-hyun
We recently hit this issue on our CI pipeline. The solution was upgrading roxygen2 to 7.3.3.

dongjoon-hyun · 2026-03-17T06:56:37Z

Thank you. Could you make an independent PR, @sarutak ? I hope we can recover the CI in any way. We can try concurrently.

dongjoon-hyun · 2026-03-17T06:58:37Z

If someone recovers the CI, we can close this one.

sarutak · 2026-03-17T07:18:01Z

Thank you. Could you make an independent PR, @sarutak ? I hope we can recover the CI in any way. We can try concurrently.

OK, I've opened a PR with another approach (#54855).
But let's see if this PR can fix the issue.

dongjoon-hyun · 2026-03-17T07:20:19Z

Thank you, @sarutak !

dongjoon-hyun · 2026-03-17T07:39:39Z

To @sarutak , this PR recovered the CIs for both.

I'm monitoring the next step, sparkr.

sarutak · 2026-03-17T07:53:00Z

@dongjoon-hyun
roxygen2 7.3.3 seems used even in this PR.
https://github.com/dongjoon-hyun/spark/actions/runs/23182231424/job/67357670225#step:10:3657

So, I'll close my PR.

dongjoon-hyun · 2026-03-17T07:55:57Z

Got it~

sarutak · 2026-03-17T08:11:44Z

Maybe, we need to update the following files later as well in a follow-up PR.

dongjoon-hyun · 2026-03-17T08:14:20Z

Ya, I agree with you. I just want to keep them separately because CI recovery is only the first step. We may need to spend more time for the investigation the actual version of Roxygen and others. The current SparkR environment seems messy because it's abandoned for a long time.

This PR only aims to recover PR builder and commit builder on the master branch. All the other daily CIs or release branches and spark-rm release script should be handled carefully one by one.

sarutak · 2026-03-17T08:18:28Z

I have no idea why just upgrading roxygen2 in our CI environment addressed the issue but the setting is different from GA workflow, which might cause the difference.

dongjoon-hyun · 2026-03-17T08:25:49Z

Thank you, @LuciferYang and @sarutak .

sarutak

LGTM if GA passes.

sarutak

GA passed. Merging to master.
Thank you @dongjoon-hyun and @LuciferYang !

dongjoon-hyun · 2026-03-17T14:51:36Z

Thank you, @sarutak and @LuciferYang !

…e/(lint|docs|sparkr)/Dockerfile` This PR aims to recover the CI by installing `remotes` R package for the following Dockerfiles. - dev/spark-test-image/lint/Dockerfile - dev/spark-test-image/docs/Dockerfile - dev/spark-test-image/sparkr/Dockerfile Currently, `master` branch is broken due to the new `devtools` 2.5.0 release. This PR aims to installation `remotes` package explicitly because we cannot pin `devtools` itself. - https://github.com/r-lib/devtools/releases/tag/v2.5.0 (2026-03-16) <img width="754" height="263" alt="Screenshot 2026-03-16 at 23 57 25" src="https://github.com/user-attachments/assets/c5b927a4-fe02-490a-866d-1f43460499f2" /> https://github.com/apache/spark/actions/runs/23180660599/job/67352803326 ``` 801.6 Error in devtools::install_version("pkgdown", version = "2.0.1", repos = "https://cloud.r-project.org/") : 801.6 The package "remotes" is required. ``` No. Recover the CIs. Generated-by: Claude Code (claude-opus-4-6) Closes #54853 from dongjoon-hyun/SPARK-56025. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Kousuke Saruta <[email protected]> (cherry picked from commit 54164e9) Signed-off-by: Dongjoon Hyun <[email protected]>

dongjoon-hyun · 2026-03-17T16:34:19Z

I backported this to branch-4.1 to recover branch-4.1 CI.

…e/(lint|docs|sparkr)/Dockerfile` This PR aims to recover the CI by installing `remotes` R package for the following Dockerfiles. - dev/spark-test-image/lint/Dockerfile - dev/spark-test-image/docs/Dockerfile - dev/spark-test-image/sparkr/Dockerfile Currently, `master` branch is broken due to the new `devtools` 2.5.0 release. This PR aims to installation `remotes` package explicitly because we cannot pin `devtools` itself. - https://github.com/r-lib/devtools/releases/tag/v2.5.0 (2026-03-16) <img width="754" height="263" alt="Screenshot 2026-03-16 at 23 57 25" src="https://github.com/user-attachments/assets/c5b927a4-fe02-490a-866d-1f43460499f2" /> https://github.com/apache/spark/actions/runs/23180660599/job/67352803326 ``` 801.6 Error in devtools::install_version("pkgdown", version = "2.0.1", repos = "https://cloud.r-project.org/") : 801.6 The package "remotes" is required. ``` No. Recover the CIs. Generated-by: Claude Code (claude-opus-4-6) Closes #54853 from dongjoon-hyun/SPARK-56025. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Kousuke Saruta <[email protected]> (cherry picked from commit 54164e9) Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 8551960) Signed-off-by: Dongjoon Hyun <[email protected]>

dongjoon-hyun · 2026-03-17T18:26:37Z

This is backported to branch-4.0 too.

### What changes were proposed in this pull request? See #54872 and #54853 ### Why are the changes needed? See #54872 and #54853 ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? I am testing it in my fork. see https://github.com/HyukjinKwon/spark/actions/runs/24021551482/job/70051292787 ### Was this patch authored or co-authored using generative AI tooling? No. Closes #55217 from HyukjinKwon/SPARK-56363. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>

### What changes were proposed in this pull request? See #54872 and #54853 ### Why are the changes needed? See #54872 and #54853 ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? I am testing it in my fork. see https://github.com/HyukjinKwon/spark/actions/runs/24021551482/job/70051292787 ### Was this patch authored or co-authored using generative AI tooling? No. Closes #55217 from HyukjinKwon/SPARK-56363. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]> (cherry picked from commit 3fb32fd) Signed-off-by: Hyukjin Kwon <[email protected]>

[SPARK-56025] Install remotes R package for spark-test-image/lint/D…

65abf63

…ockerfile

dongjoon-hyun changed the title ~~[SPARK-56025][INFRA] Install remotes R package for spark-test-image/lint/Dockerfile~~ [SPARK-56025][INFRA] Install remotes R package for spark-test-image/lint/Dockerfile Mar 17, 2026

dongjoon-hyun commented Mar 17, 2026

View reviewed changes

Update docs too

d9ad4fa

dongjoon-hyun changed the title ~~[SPARK-56025][INFRA] Install remotes R package for spark-test-image/lint/Dockerfile~~ [SPARK-56025][INFRA] Install remotes R package for spark-test-image/(lint|docs)/Dockerfile Mar 17, 2026

dongjoon-hyun marked this pull request as draft March 17, 2026 07:02

dongjoon-hyun marked this pull request as ready for review March 17, 2026 07:32

dongjoon-hyun mentioned this pull request Mar 17, 2026

[SPARK-56027][INFRA] Fix NoEmptyContinuation warning of spark-test-image/lint/Dockerfile #54856

Closed

Update sparkr too

ed7eb5f

dongjoon-hyun changed the title ~~[SPARK-56025][INFRA] Install remotes R package for spark-test-image/(lint|docs)/Dockerfile~~ [SPARK-56025][INFRA] Install remotes R package for spark-test-image/(lint|docs|sparkr)/Dockerfile Mar 17, 2026

This was referenced Mar 17, 2026

[SPARK-56013][BUILD] Upgrade JAXB to 4.0.6 #54835

Closed

[SPARK-55995][SQL] Support TIMESTAMP WITH LOCAL TIME ZONE in SQL syntax #54813

Closed

[SPARK-56000][BUILD] Upgrade arrow-java to 19.0.0 #54820

Closed

LuciferYang approved these changes Mar 17, 2026

View reviewed changes

sarutak approved these changes Mar 17, 2026

View reviewed changes

sarutak closed this in 54164e9 Mar 17, 2026

dongjoon-hyun deleted the SPARK-56025 branch March 17, 2026 14:51

HyukjinKwon mentioned this pull request Apr 6, 2026

[SPARK-56363][INFRA] Add remotes in Spark release image #55217

Closed

Conversation

dongjoon-hyun commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

dongjoon-hyun Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

sarutak commented Mar 17, 2026

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

sarutak commented Mar 17, 2026

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

sarutak commented Mar 17, 2026

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

sarutak commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dongjoon-hyun commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sarutak commented Mar 17, 2026

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

sarutak left a comment

Choose a reason for hiding this comment

Uh oh!

sarutak left a comment

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

dongjoon-hyun commented Mar 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dongjoon-hyun commented Mar 17, 2026 •

edited

Loading

sarutak commented Mar 17, 2026 •

edited

Loading

dongjoon-hyun commented Mar 17, 2026 •

edited

Loading