Update path rewriting to support setuptools v64's PEP660 editable install mode#1357
Merged
Update path rewriting to support setuptools v64's PEP660 editable install mode#1357
Conversation
Malax
approved these changes
Aug 17, 2022
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Heroku, the application source directory exists at a different path at build time (
/tmp/build_<hash>), than it does at runtime (/app). As such, the buildpack has to perform path rewriting via.profile.dscripts at runtime, to ensure any packaging related absolute paths in the build output are rewritten to reference the new path. (Thankfully this awful path rewriting will no longer be necessary in the future with CNBs.)Previously the only files this path rewriting needed to update were the
*.pthand*.egg-linkfiles insite-packagescreated by setuptools when performing editable installs.However setuptools v64 added support for PEP660 based editable install hooks:
https://setuptools.pypa.io/en/latest/history.html#v64-0-0
https://peps.python.org/pep-0660/
This feature is only used for projects that have a
pyproject.toml, and for such projects, if the config is deemed complex enough, setuptools creates a new finder script insite-packagesthat dynamically handles package resolution. (Simpler configs get a static.pthfile, which works fine with our existing path rewriting.)This new file embeds the absolute path of the source directory at build time, so must be rewritten too. It has a filename of form:
__editable___my_package_0_0_1_finder.pyAs such, this PR adds support for rewriting these files, along with updated test fixtures to provide coverage of
pyproject.tomlbased editable installs (alongside the existingsetup.pybased test fixture).Whilst writing the new test, I encountered a difference in behaviour with setuptool's new editable install mode, which meant the fixtures had to be nested inside a
packages/directory in order to avoid anImportErrordue to the fact that the Python buildpack currently setsPYTHONPATH=/appat runtime. See:pypa/setuptools#3535
Note:
pyproject.tomlpip uses the approach described in PEP518, which uses an isolated build environment. This environment typically pulls in latest setuptools (though this is controllable by the package owner via[build-system]inpyproject.toml), overriding our pinned global setuptools install.A big thanks to @mrcljx for the initial PR in #1355 on which this was based :-)
Closes #1355.
GUS-W-11608693.