Migrate away from the deprecated pkgutil.find_loader()#1493
Conversation
In the newly released Python 3.12, the `pkgutil.find_loader()` API was deprecated: https://docs.python.org/3.12/library/pkgutil.html#pkgutil.find_loader It's been replaced by `importlib.util.find_spec` which is supported since Python 3.4 (which is fine for us to use now, but wasn't an option back when `is_module_available()` was written, since at that time the buildpack still had to support Python 2.7): https://docs.python.org/3.12/library/importlib.html#importlib.util.find_spec This API was used by the buildpack to determine whether certain packages (like `django` or `nltk`) are installed, and so resulted in deprecation warnings being output in the build log: ``` <string>:1: DeprecationWarning: 'pkgutil.find_loader' is deprecated and slated for removal in Python 3.14; use importlib.util.find_spec() instead ``` These deprecation warnings also prevented the output from `bin/release` being parsed correctly for Django apps, resulting in: ``` failed to read buildpack metadata: (<unknown>): mapping values are not allowed in this context at line 1 column 31 ``` These can be seen in the Heroku CI tests and Review App build logs of this PR which upgrades the Python getting started guide app to Python 3.12: heroku/python-getting-started#209 https://dashboard.heroku.com/pipelines/4826c212-2bb2-4882-a147-4d0501eee7a4/tests/190 https://dashboard.heroku.com/apps/getting-star-edmorley-p-biivve/activity/builds/a6feca96-d451-4f01-b66d-c5d2185d5107 GUS-W-14230170.
edmorley
force-pushed
the
python-3.12-deprecation
branch
from
14f5b96 to
90014a0
Compare
schneems
left a comment
schneems
left a comment
There was a problem hiding this comment.
Is the output of DeprecationWarning consistent in Python? We could add a refutation to integration tests to catch them earlier.
In the newly released Python 3.12, the `pkgutil.find_loader()` API was deprecated: https://docs.python.org/3.12/library/pkgutil.html#pkgutil.find_loader It's been replaced by `importlib.util.find_spec` which is supported since Python 3.4 (which is fine for us to use now, but wasn't an option back when `is_module_available()` was written, since at that time the buildpack still had to support Python 2.7): https://docs.python.org/3.12/library/importlib.html#importlib.util.find_spec This API was used by the buildpack to determine whether certain packages (like `django` or `nltk`) are installed, and so resulted in deprecation warnings being output in the build log: ``` <string>:1: DeprecationWarning: 'pkgutil.find_loader' is deprecated and slated for removal in Python 3.14; use importlib.util.find_spec() instead ``` These deprecation warnings also prevented the output from `bin/release` being parsed correctly for Django apps, resulting in: ``` failed to read buildpack metadata: (<unknown>): mapping values are not allowed in this context at line 1 column 31 ``` These can be seen in the Heroku CI tests and Review App build logs of this PR which upgrades the Python getting started guide app to Python 3.12: heroku/python-getting-started#209 https://dashboard.heroku.com/pipelines/4826c212-2bb2-4882-a147-4d0501eee7a4/tests/190 https://dashboard.heroku.com/apps/getting-star-edmorley-p-biivve/activity/builds/a6feca96-d451-4f01-b66d-c5d2185d5107 GUS-W-14230170.
|
Published: |
The tests in this repo use multi-line assertions that in many cases cover the majority of the build log output, so we'd catch spurious output in the build output in many cases (unless it appeared at the very end of the log etc). The issue here is that whilst we test the buildpack against every single supported major Python version (and on every supported stack), we only do so for the basic tests, with the more niche buildpack features only being tested against the buildpack's default Python version. Running every test against all Python versions would increase CI time considerably, in return for catching a once in 4 year type deprecation warning. For the CNB I've already stopped using Python APIs for this type of check to remove the chance of this scenario occurring. Perhaps it's one of the things I can backport (though needing to continue to support NLTK in the classic buildpack won't help, since the approach I used for Django detection can't be used for NLTK). |
2 participants
In the newly released Python 3.12, the
pkgutil.find_loader()API was deprecated:https://docs.python.org/3.12/library/pkgutil.html#pkgutil.find_loader
This API was used by the buildpack to determine whether certain packages (like
djangoornltk) are installed.It's been replaced by
importlib.util.find_specwhich has been supported since Python 3.4 (which is fine for us to use now, but wasn't an option back whenis_module_available()was written, since at that time the buildpack still had to support Python 2.7):https://docs.python.org/3.12/library/importlib.html#importlib.util.find_spec
Using the old API results in deprecation warnings being output in the build log:
These deprecation warnings also prevent the output from
bin/releasebeing parsed correctly for Django apps, resulting in:These can be seen in the Heroku CI tests and Review App build logs of this PR which upgrades the Python getting started guide app to Python 3.12:
heroku/python-getting-started#209
https://dashboard.heroku.com/pipelines/4826c212-2bb2-4882-a147-4d0501eee7a4/tests/190
https://dashboard.heroku.com/apps/getting-star-edmorley-p-biivve/activity/builds/a6feca96-d451-4f01-b66d-c5d2185d5107
GUS-W-14230170.