Skip to content

[SPARK-54263][INFRA] Fix run-in-container script to use Python 3.11 instead of 3.9#52959

Closed
dongjoon-hyun wants to merge 1 commit intoapache:masterfrom
dongjoon-hyun:SPARK-54263
Closed

[SPARK-54263][INFRA] Fix run-in-container script to use Python 3.11 instead of 3.9#52959
dongjoon-hyun wants to merge 1 commit intoapache:masterfrom
dongjoon-hyun:SPARK-54263

Conversation

@dongjoon-hyun
Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun commented Nov 10, 2025

What changes were proposed in this pull request?

This PR aims to fix run-in-container script to use Python 3.11 instead of 3.9.

Why are the changes needed?

Since Apache Spark 4.0.0, we have an optional way to build docs in Docker image.

spark/docs/README.md

Lines 129 to 136 in fa33ea0

## Build docs with docker image (Optional)
As a Spark developer, you can generate all documents locally as follows:
Note: Before running it, you need to have `docker` installed.
```sh
$ dev/spark-test-image-util/docs/build-docs

When we started to use Python 3.11 instead of 3.9 for Apache Spark 4.1.0, we missed to update this run-in-container.

Does this PR introduce any user-facing change?

No behavior change because this is an optional way to generate Spark docs.

How was this patch tested?

Manually test.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the BUILD label Nov 10, 2025
@dongjoon-hyun
Copy link
Copy Markdown
Member Author

cc @zhengruifeng

@dongjoon-hyun
Copy link
Copy Markdown
Member Author

Thank you, @zhengruifeng ! Merged to master/4.1.

dongjoon-hyun added a commit that referenced this pull request Nov 10, 2025
… instead of 3.9

### What changes were proposed in this pull request?

This PR aims to fix `run-in-container` script to use Python 3.11 instead of 3.9.

### Why are the changes needed?

Since Apache Spark 4.0.0, we have an optional way to build docs in Docker image.
- #48860

https://github.com/apache/spark/blob/fa33ea000a0bda9e5a3fa1af98e8e85b8cc5e4d4/docs/README.md?plain=1#L129-L136

When we started to use Python 3.11 instead of 3.9 for Apache Spark 4.1.0, we missed to update this `run-in-container`.
- #51150
- #51198

### Does this PR introduce _any_ user-facing change?

No behavior change because this is an optional way to generate Spark docs.

### How was this patch tested?

Manually test.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #52959 from dongjoon-hyun/SPARK-54263.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 6466ed5)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@dongjoon-hyun dongjoon-hyun deleted the SPARK-54263 branch November 10, 2025 02:34
zifeif2 pushed a commit to zifeif2/spark that referenced this pull request Nov 22, 2025
… instead of 3.9

### What changes were proposed in this pull request?

This PR aims to fix `run-in-container` script to use Python 3.11 instead of 3.9.

### Why are the changes needed?

Since Apache Spark 4.0.0, we have an optional way to build docs in Docker image.
- apache#48860

https://github.com/apache/spark/blob/996dde53f999de2ec08d7c22929d2b855b99eeab/docs/README.md?plain=1#L129-L136

When we started to use Python 3.11 instead of 3.9 for Apache Spark 4.1.0, we missed to update this `run-in-container`.
- apache#51150
- apache#51198

### Does this PR introduce _any_ user-facing change?

No behavior change because this is an optional way to generate Spark docs.

### How was this patch tested?

Manually test.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#52959 from dongjoon-hyun/SPARK-54263.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
huangxiaopingRD pushed a commit to huangxiaopingRD/spark that referenced this pull request Nov 25, 2025
… instead of 3.9

### What changes were proposed in this pull request?

This PR aims to fix `run-in-container` script to use Python 3.11 instead of 3.9.

### Why are the changes needed?

Since Apache Spark 4.0.0, we have an optional way to build docs in Docker image.
- apache#48860

https://github.com/apache/spark/blob/fa33ea000a0bda9e5a3fa1af98e8e85b8cc5e4d4/docs/README.md?plain=1#L129-L136

When we started to use Python 3.11 instead of 3.9 for Apache Spark 4.1.0, we missed to update this `run-in-container`.
- apache#51150
- apache#51198

### Does this PR introduce _any_ user-facing change?

No behavior change because this is an optional way to generate Spark docs.

### How was this patch tested?

Manually test.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#52959 from dongjoon-hyun/SPARK-54263.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants