[SPARK-56313][PYTHON][FOLLOWUP] Remove the generic for rddsampler methods#55144
[SPARK-56313][PYTHON][FOLLOWUP] Remove the generic for rddsampler methods#55144gaogaotiantian wants to merge 1 commit intoapache:masterfrom
Conversation
|
Thanks! merging to master. |
|
another failure ... why can't PR GHA capture it? https://github.com/apache/spark/actions/runs/23874309148/job/69613658492 |
|
@pan3793 it seems the this job |
should they be aligned? or it's by design? |
|
These are two different workflows that use different versions of Python. The |
|
inconsistency comes here, #54310 upgraded Python from 3.11 to 3.12 for doc build |
|
Yeah, but even if the doc CI is consistent, we would've broken the scheduled 3.10/3.11 CI. We can't catch all issues with python compatibilities before merging. Catching it the next day is not too bad. |
|
We test with Python 3.11 on the Apache Arrow nightlies integration job with Spark. Just sharing that I was here looking for this :) Our nightlies trigger here: Maybe a minimal 3.10 CI validation when there are changes on Python files could be worth it? Just to validate there are no breaking changes for the oldest supported Python? @gaogaotiantian @HyukjinKwon |
|
We are limited by the number of concurrent tasks per PR. That's ASF policy. Switching to another version of Python is definitely not impossible, but kind of conflict with our current testing framework - with all the docker images and everything. |
What changes were proposed in this pull request?
Remove the generic (
[T]) for rddsampler methodsWhy are the changes needed?
It's supported starting from 3.12 but we support python since 3.10.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
mypy passed locally.
Was this patch authored or co-authored using generative AI tooling?
No.