Commit 40f3ea7
committed
[SPARK-52927][PYTHON][INFRA] Upgrade pyarrow to 15.0 in image
### What changes were proposed in this pull request?
Upgrade pyarrow to 15.0 in image `python-minimum`
I will test pyarrow version for `python-ps-minimum`, and finally update the docs, in separate PRs.
### Why are the changes needed?
recent changes used features in newer versions, e.g.
1. `pa.Table.from_struct_array` used in 537f813 is only available since 15.0
2. `pa.array(list of pa.Scalar)` used in arrow UDF, is only available since 13.0
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
PR builder with
```
default: '{"PYSPARK_IMAGE_TO_TEST": "python-minimum", "PYTHON_TO_TEST": "python3.10"}'
```
https://github.com/zhengruifeng/spark/runs/46547147080
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #51632 from zhengruifeng/fix_arrow_old_version.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>python-minimum
1 parent 79ba12a commit 40f3ea7
1 file changed
+1
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | | - | |
| 64 | + | |
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| |||
0 commit comments