Skip to content

[SPARK-51627][INFRA][PYTHON] Add a scheduled workflow for numpy 2.1.3#50426

Closed
zhengruifeng wants to merge 2 commits intoapache:masterfrom
zhengruifeng:infra_more_env
Closed

[SPARK-51627][INFRA][PYTHON] Add a scheduled workflow for numpy 2.1.3#50426
zhengruifeng wants to merge 2 commits intoapache:masterfrom
zhengruifeng:infra_more_env

Conversation

@zhengruifeng
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Add a scheduled (every 3 days) workflow for numpy 2.1.3

Why are the changes needed?

we see many 3-rd packages not compatible with numpy 2.2, e.g.

tensorflow only supports numpy<2.2.0 
ydata-profiling 4.16.0 depends on numpy<2.2 and >=1.16.0
numba 0.61.0 depends on numpy<2.2 and >=1.24

Add this scheduled workflow to make sure pyspark works with numpy<2.2

Does this PR introduce any user-facing change?

no, infra-only

How was this patch tested?

PR builder with

default: '{"PYSPARK_IMAGE_TO_TEST": "numpy-213", "PYTHON_TO_TEST": "python3.11"}'

Was this patch authored or co-authored using generative AI tooling?

no

@zhengruifeng
Copy link
Copy Markdown
Contributor Author

merged to master

@zhengruifeng zhengruifeng deleted the infra_more_env branch March 27, 2025 08:25
zhengruifeng added a commit that referenced this pull request Mar 28, 2025
followup of #50426, the numpy version is 2.1.3 instead of 3.1.2

Closes #50445 from zhengruifeng/fix_numpy_job_name.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants