PoC: update PythonPackage easyblock to build wheel first when installing Python package with pip#3068
Closed
Flamefire wants to merge 2 commits intoeasybuilders:developfrom
Closed
Conversation
PythonPackage eaysyblock to build wheel first when installing Python package with pip
PythonPackage eaysyblock to build wheel first when installing Python package with pipPythonPackage easyblock to build wheel first when installing Python package with pip
3b1123c to
30fb1bc
Compare
15680d1 to
d487e9f
Compare
d487e9f to
36349b5
Compare
Check that replacement actually happens and wheel files are created. Also change config values for following install step(s).
36349b5 to
144af11
Compare
Contributor
Author
|
Closing as the added complexity is not worth the gain |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
(created using
eb --new-pr)This is a Proof of Concept only.
It adds the option to set
use_pip = 'wheel'which matches more our build-test-install cycle.Currently when using pip we have this:
pip install --prefix /tmp/foo .pip install --prefix /final/foo .Confusingly all build options need to be passed to
(pre)installoptsand as Python usually creates a "wheel file" (basically a compiled, installable zip) this duplicates work.This option when set does the following:
pip wheel --wheel-dir /new-tmp-dirpip install --prefix /tmp/foo /new-tmp-dir/*.whlpip install --prefix /final/foo /new-tmp-dir/*.whlThis avoids duplicate work and ensures that what is tested is what gets installed (currently we build twice which could yield different results). It is also nice to have separate timing and output for the actual build
However it turned out a bit more involved than anticipated. E.g. not all options for
pip installapply topip wheeland vice versa. Also the time savings are not that great. I tested PyTorch 2.1.2 on a (admittedly high-cpu) machine. And after an initial build/install the times are:setup.py install: ~35spip install .: ~63spip install *.whl: ~17sSo the time saving is around 11s which IMO isn't worth the added complexity.
However it brings back a dedicated build step, which shows up as such in EB status/progress and not confusingly in the test or install step.