BLD: Try using shared memory utilities in Cython to reduce wheel sizes#61384
BLD: Try using shared memory utilities in Cython to reduce wheel sizes#61384lithomas1 merged 10 commits intopandas-dev:mainfrom
Conversation
|
around 10% savings in both compressed/uncompressed size scikit-learn seems to have gotten a lot more mileage out of this optimization... |
| if cy.version().version_compare('>=3.1.0') | ||
| add_project_arguments('-Xfreethreading_compatible=true', language: 'cython') | ||
|
|
||
| # Use shared utility code to reduce wheel sizes |
There was a problem hiding this comment.
Cool seems useful. Is there any discussion to add this natively to Meson instead of custom-coding this ourselves?
There was a problem hiding this comment.
I don't see how that's possible, since it generates an extra extension module and projects may have their code set up (e.g., with a Meson generator) to apply Cython/C compile args to all their extension modules. They may also have opinions about where the extension module should live in the import tree.
| "meson>=1.2.1,<2", | ||
| "wheel", | ||
| "Cython~=3.0.5", # Note: sync with setup.py, environment.yml and asv.conf.json | ||
| "Cython==3.1.0rc1", # Note: sync with setup.py, environment.yml and asv.conf.json |
There was a problem hiding this comment.
You don't want to do this, no need to force it since 3.1.0 is brand new (users and distros may have reasons not to upgrade, and there may be bugs).
You can use this instead:
if cy.version().version_compare('>=3.1.0')
cython_args += ['-Xfreethreading_compatible=True']The way I did it in scipy/scipy#22796 is a bit more concise than in this PR, also with cython_args: passing in a single place.
There was a problem hiding this comment.
Actually you do have the cy.version().version_compare('>=3.1.0') already, so just no need to pin Cython. You'll get 3.1.0 anyway by default now.
|
This looks pretty good as is, and will definitely be worthwhile to do - it's a major gain in binary size for only 20 lines of code or so. It should work now, maybe retrigger CI after dropping the Cython pin? |
|
pre-commit.ci autofix |
|
Owee, I'm MrMeeseeks, Look at me. There seem to be a conflict, please backport manually. Here are approximate instructions:
And apply the correct labels and milestones. Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon! Remember to remove the If these instructions are inaccurate, feel free to suggest an improvement. |
|
Probably ok to punt this to 3.0 since that's happening soon and to minimize risk for 2.3. |
pandas-dev#61384) * BLD: Try using shared memory utilities in Cython to reduce wheel sizes * fix syntax * maybe fix? * switch keyword order? * try putting as list? * try the cython rc * add more version checks * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
3 participants
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.