Skip to content

Update pip to 22.2.2 and setuptools to 63.4.3#1344

Merged
edmorley merged 1 commit intomainfrom
update-pip-setuptools
Aug 15, 2022
Merged

Update pip to 22.2.2 and setuptools to 63.4.3#1344
edmorley merged 1 commit intomainfrom
update-pip-setuptools

Conversation

@edmorley
Copy link
Copy Markdown
Member

@edmorley edmorley commented Aug 14, 2022

Updates (for Python 3.7+ only):

  • pip from 22.1.2 to 22.2.2
  • setuptools from 60.10.0 to 63.4.3

Changelogs:
https://pip.pypa.io/en/stable/news/#v22-2-2
https://setuptools.pypa.io/en/latest/history.html#v63-4-3

Full changes:
pypa/pip@21.1.2...22.2.2
pypa/setuptools@v60.10.0...v63.4.3

Pip wheel uploaded using:

$ pip download --no-cache pip==22.2.2
...
Saved ./pip-22.2.2-py3-none-any.whl
Successfully downloaded pip
$ aws s3 sync . s3://heroku-buildpack-python/common/ --exclude "*" --include "pip-*.whl" --profile python --dryrun
(dryrun) upload: ./pip-22.2.2-py3-none-any.whl to s3://heroku-buildpack-python/common/pip-22.2.2-py3-none-any.whl
$ aws s3 sync . s3://heroku-buildpack-python/common/ --exclude "*" --include "pip-*.whl" --profile python
upload: ./pip-22.2.2-py3-none-any.whl to s3://heroku-buildpack-python/common/pip-22.2.2-py3-none-any.whl

GUS-W-11592987.
GUS-W-11592988.

@edmorley edmorley self-assigned this Aug 14, 2022
@edmorley edmorley marked this pull request as ready for review August 14, 2022 09:24
@edmorley edmorley requested a review from a team as a code owner August 14, 2022 09:24
@edmorley edmorley force-pushed the update-pip-setuptools branch from 72ccd35 to b765990 Compare August 15, 2022 11:27
@edmorley edmorley enabled auto-merge (squash) August 15, 2022 11:27
@edmorley edmorley merged commit 4646cff into main Aug 15, 2022
@edmorley edmorley deleted the update-pip-setuptools branch August 15, 2022 11:29
@edmorley
Copy link
Copy Markdown
Member Author

pull bot pushed a commit to eloisetwaine/heroku-buildpack-python that referenced this pull request Aug 17, 2022
…tall mode (heroku#1357)

On Heroku, the application source directory exists at a different path at build time (`/tmp/build_<hash>`), than it does at runtime (`/app`). As such, the buildpack has to perform path rewriting via `.profile.d` scripts at runtime, to ensure any packaging related absolute paths in the build output are rewritten to reference the new path. (Thankfully this awful path rewriting will no longer be necessary in the future with CNBs.)

Previously the only files this path rewriting needed to update were the `*.pth` and `*.egg-link` files in `site-packages` created by setuptools when performing editable installs.

However setuptools v64 added support for PEP660 based editable install hooks:
https://setuptools.pypa.io/en/latest/history.html#v64-0-0
https://peps.python.org/pep-0660/

This feature is only used for projects that have a `pyproject.toml`, and for such projects, [if the config is deemed complex enough](https://github.com/pypa/setuptools/blob/d03da04e024ad4289342077eef6de40013630a44/setuptools/command/editable_wheel.py#L359-L368), setuptools creates a new [finder script](https://github.com/pypa/setuptools/blob/23d455c532fca91e6f00aa5950000739b058b6e5/setuptools/command/editable_wheel.py#L740-L809) in `site-packages` that dynamically handles package resolution. (Simpler configs get a static `.pth` file, which works fine with our existing path rewriting.)

This new file embeds the absolute path of the source directory at build time, so must be rewritten too. It has a filename of form: `__editable___my_package_0_0_1_finder.py`

As such, this PR adds support for rewriting these files, along with updated test fixtures to provide coverage of `pyproject.toml` based editable installs (alongside the existing `setup.py` based test fixture).

Whilst writing the new test, I encountered a difference in behaviour with setuptool's new editable install mode, which meant the fixtures had to be nested inside a `packages/` directory in order to avoid an `ImportError` due to the fact that the Python buildpack currently sets `PYTHONPATH=/app` at runtime. See:
pypa/setuptools#3535

Note:
- The Python buildpack doesn't yet globally install this newer setuptools v64 release, since by design it pins to a specific version to prevent upstream changes from breaking apps overnight. (The version was recently updated to 63.4.3 in heroku#1344.)
- However, for packages that have a `pyproject.toml` pip uses the approach described in PEP518, which uses an [isolated build environment](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation). This environment typically pulls in latest setuptools (though this is controllable by the package owner via `[build-system]` in `pyproject.toml`), overriding our pinned global setuptools install.
- As such, even though we're not using v64 globally, users can still be broken by the upstream release.

A big thanks to @mrcljx for the initial PR in heroku#1355 on which this was based :-)

Closes heroku#1355.
GUS-W-11608693.

Co-authored-by: Marcel Jackwerth <marceljackwerth@gmail.com>
edmorley added a commit that referenced this pull request Sep 28, 2022
Since it was linking to an unrelated PR.
edmorley added a commit that referenced this pull request Sep 28, 2022
Since it was linking to an unrelated PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants