Skip to content

Commit daf9fe3

Browse files
committed
[SPARK-49862][INFRA][PYTHON] Restore package installations for Python 3.13 daily builder
### What changes were proposed in this pull request? Restore package installations for Python 3.13 daily builder ### Why are the changes needed? some tests were skipped due to missed installations ### Does this PR introduce _any_ user-facing change? no, test-only ### How was this patch tested? PR builder with env: ``` default: '{"PYSPARK_IMAGE_TO_TEST": "python-313", "PYTHON_TO_TEST": "python3.13"}' ``` https://github.com/zhengruifeng/spark/actions/runs/14333711996/job/40175722749 ### Was this patch authored or co-authored using generative AI tooling? no Closes #50539 from zhengruifeng/infra_py313. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
1 parent 367d206 commit daf9fe3

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

dev/spark-test-image/python-313/Dockerfile

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,12 +68,13 @@ RUN apt-get update && apt-get install -y \
6868

6969

7070
ARG BASIC_PIP_PKGS="numpy pyarrow>=19.0.0 six==1.16.0 pandas==2.2.3 scipy plotly<6.0.0 mlflow>=2.8.1 coverage matplotlib openpyxl memory-profiler>=0.61.0 scikit-learn>=1.3.2"
71+
# Python deps for Spark Connect
7172
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.29.1 googleapis-common-protos==1.65.0 graphviz==0.20.3"
7273

73-
7474
# Install Python 3.13 packages
7575
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.13
76-
# TODO(SPARK-49862) Add BASIC_PIP_PKGS and CONNECT_PIP_PKGS to Python 3.13 image when it supports Python 3.13
7776
RUN python3.13 -m pip install --ignore-installed blinker>=1.6.2 # mlflow needs this
78-
RUN python3.13 -m pip install numpy>=2.1 pyarrow>=19.0.0 six==1.16.0 pandas==2.2.3 scipy coverage matplotlib openpyxl grpcio==1.67.0 grpcio-status==1.67.0 lxml jinja2 && \
77+
RUN python3.13 -m pip install $BASIC_PIP_PKGS unittest-xml-reporting $CONNECT_PIP_PKGS lxml && \
78+
python3.13 -m pip install 'torch<2.6.0' torchvision --index-url https://download.pytorch.org/whl/cpu && \
79+
python3.13 -m pip install torcheval && \
7980
python3.13 -m pip cache purge

0 commit comments

Comments
 (0)