Add support for the Heroku-22 stack

Adds binary generation and testing support for Heroku-22.

Runtime version support for Heroku-22 is as follows:
* Python 3.6 and older: Not supported since EOL.
* Python 3.7/3.8: Not supported since:
   - they are in the security-fix-only phase of their lifecycle
   - they reach EOL in 2023-06 and 2024-10 respectively, which is well
     before the EOL date of Heroku-22
   - upgrading from 3.7/3.9 to 3.9 is in most cases trivial, and
     so we should be encouraging users to upgrade.
* Python 3.9: Only the current latest patch release (3.9.12) supported
   (plus any future 3.9 releases).
* Python 3.10: Only the current latest patch release (3.10.4) supported
   (plus any future 3.10 releases).
* PyPy: Not supported, since is deprecated, was only ever a beta
   feature, the PyPy versions available for Heroku-18/20 are
   significantly out of date, and usage is virtually non-existent.

GUS-W-10343718.

Author: edmorley

.circleci/config.yml

@@ -47,7 +47,7 @@ jobs:
     parameters:
       stack:
         type: enum
-        enum: ["heroku-18", "heroku-20"]
+        enum: ["heroku-18", "heroku-20", "heroku-22"]
     docker:
       - image: cimg/ruby:2.7
     working_directory: /mnt/ramdisk/project
@@ -87,4 +87,4 @@ workflows:
       - hatchet:
           matrix:
             parameters:
-              stack: ["heroku-18", "heroku-20"]
+              stack: ["heroku-18", "heroku-20", "heroku-22"]

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## Unreleased
 
+- Add support for Heroku-22 ([#1299](https://github.com/heroku/heroku-buildpack-python/pull/1299)).
 - Use shared builds + LTO when building Python 3.10 binaries ([#1320](https://github.com/heroku/heroku-buildpack-python/pull/1320)).
   Note: This and the other Python binary changes below will only take effect for future Python
   version releases (or future Heroku stacks) - existing Python binaries are not being recompiled.

Makefile

@@ -1,8 +1,8 @@
 # These targets are not files
 .PHONY: lint lint-scripts lint-ruby compile builder-image buildenv deploy-runtimes publish
 
-STACK ?= heroku-20
-STACKS ?= heroku-18 heroku-20
+STACK ?= heroku-22
+STACKS ?= heroku-18 heroku-20 heroku-22
 PLATFORM := linux/amd64
 FIXTURE ?= spec/fixtures/python_version_unspecified
 ENV_FILE ?= builds/dockerenv.default

README.md

@@ -60,7 +60,7 @@ Specify a Python Runtime
 
 Supported runtime options include:
 
-- `python-3.10.4`
-- `python-3.9.12`
-- `python-3.8.13`
-- `python-3.7.13`
+- `python-3.10.4` on all [supported stacks](https://devcenter.heroku.com/articles/stack#stack-support-details)
+- `python-3.9.12` on all [supported stacks](https://devcenter.heroku.com/articles/stack#stack-support-details)
+- `python-3.8.13` on Heroku-18 and Heroku-20 only
+- `python-3.7.13` on Heroku-18 and Heroku-20 only

builds/README.md

@@ -74,7 +74,7 @@ make deploy-runtimes RUNTIMES='python-X.Y.Z' STACKS='heroku-18'
 Multiple runtimes can also be specified (useful for when adding a new stack), like so:
 
 ```bash
-make deploy-runtimes RUNTIMES='python-A.B.C python-X.Y.Z' STACKS='heroku-20'
+make deploy-runtimes RUNTIMES='python-A.B.C python-X.Y.Z' STACKS='heroku-22'
 ```
 
 Note: Both `RUNTIMES` and `STACKS` are space delimited.

builds/heroku-22.Dockerfile

@@ -0,0 +1,20 @@
+FROM heroku/heroku:22-build
+
+ENV WORKSPACE_DIR="/app/builds" \
+    S3_BUCKET="heroku-buildpack-python" \
+    S3_PREFIX="heroku-22/" \
+    STACK="heroku-22"
+
+RUN apt-get update \
+    && DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
+        libsqlite3-dev \
+        python3-pip \
+        python3-setuptools \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+COPY requirements.txt /app/
+RUN pip3 install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
+
+COPY . /app

builds/runtimes/python

@@ -12,6 +12,12 @@ VERSION=$(echo "${FORMULA_FILENAME}" | cut --delimiter '-' --fields 2)
 
 echo "Building Python ${VERSION}..."
 
+if [[ "${STACK}" != "heroku-18" && "${STACK}" != "heroku-20" && "${VERSION}" == 3.[7-8].* ]]; then
+  echo "Python 3.7 and 3.8 are only supported on Heroku-20 and older!" >&2
+  echo "Override the default stacks list using: STACKS='heroku-18 heroku-20'" >&2
+  exit 1
+fi
+
 # See: https://www.python.org/downloads/ -> "OpenPGP Public Keys"
 case "${VERSION}" in
   3.10.*)

spec/hatchet/pipenv_spec.rb

@@ -20,6 +20,19 @@
   end
 end
 
+RSpec.shared_examples 'aborts the build with a runtime not available message (Pipenv)' do |requested_version|
+  it 'aborts the build with a runtime not available message' do
+    app.deploy do |app|
+      expect(clean_output(app.output)).to include(<<~OUTPUT)
+        remote: -----> Python app detected
+        remote: -----> Using Python version specified in Pipfile.lock
+        remote:  !     Requested runtime (python-#{requested_version}) is not available for this stack (#{app.stack}).
+        remote:  !     Aborting.  More info: https://devcenter.heroku.com/articles/python-support
+      OUTPUT
+    end
+  end
+end
+
 RSpec.describe 'Pipenv support' do
   context 'without a Pipfile.lock' do
     let(:app) { Hatchet::Runner.new('spec/fixtures/pipenv_no_lockfile') }
@@ -85,20 +98,11 @@
       end
     end
 
-    context 'when using Heroku-20', stacks: %w[heroku-20] do
+    context 'when using Heroku-20 or newer', stacks: %w[heroku-20 heroku-22] do
       let(:allow_failure) { true }
 
-      it 'fails the build' do
-        # Python 2.7 is EOL, so it has not been built for Heroku-20.
-        app.deploy do |app|
-          expect(clean_output(app.output)).to include(<<~OUTPUT)
-            remote: -----> Python app detected
-            remote: -----> Using Python version specified in Pipfile.lock
-            remote:  !     Requested runtime (python-#{LATEST_PYTHON_2_7}) is not available for this stack (#{app.stack}).
-            remote:  !     Aborting.  More info: https://devcenter.heroku.com/articles/python-support
-          OUTPUT
-        end
-      end
+      # Python 2.7 is EOL, so has not been built for newer stacks.
+      include_examples 'aborts the build with a runtime not available message (Pipenv)', LATEST_PYTHON_2_7
     end
   end
 
@@ -112,34 +116,64 @@
   end
 
   context 'with a Pipfile.lock containing python_version 3.6' do
-    let(:app) { Hatchet::Runner.new('spec/fixtures/pipenv_python_3.6') }
+    let(:allow_failure) { false }
+    let(:app) { Hatchet::Runner.new('spec/fixtures/pipenv_python_3.6', allow_failure: allow_failure) }
 
-    it 'builds with the latest Python 3.6' do
-      app.deploy do |app|
-        expect(clean_output(app.output)).to match(Regexp.new(<<~REGEX))
-          remote: -----> Python app detected
-          remote: -----> Using Python version specified in Pipfile.lock
-          remote: cp: cannot stat '/tmp/build_.*/requirements.txt': No such file or directory
-          remote: -----> Installing python-#{LATEST_PYTHON_3_6}
-          remote: -----> Installing pip 21.3.1, setuptools 59.6.0 and wheel 0.37.1
-          remote: -----> Installing dependencies with Pipenv 2020.11.15
-          remote:        Installing dependencies from Pipfile.lock \\(.*\\)...
-          remote: -----> Installing SQLite3
-        REGEX
+    context 'when using Heroku-18 or Heroku-20', stacks: %w[heroku-18 heroku-20] do
+      it 'builds with the latest Python 3.6' do
+        app.deploy do |app|
+          expect(clean_output(app.output)).to match(Regexp.new(<<~REGEX))
+            remote: -----> Python app detected
+            remote: -----> Using Python version specified in Pipfile.lock
+            remote: cp: cannot stat '/tmp/build_.*/requirements.txt': No such file or directory
+            remote: -----> Installing python-#{LATEST_PYTHON_3_6}
+            remote: -----> Installing pip 21.3.1, setuptools 59.6.0 and wheel 0.37.1
+            remote: -----> Installing dependencies with Pipenv 2020.11.15
+            remote:        Installing dependencies from Pipfile.lock \\(.*\\)...
+            remote: -----> Installing SQLite3
+          REGEX
+        end
       end
     end
+
+    context 'when using Heroku-22', stacks: %w[heroku-22] do
+      let(:allow_failure) { true }
+
+      # Python 3.6 is EOL, so has not been built for newer stacks.
+      include_examples 'aborts the build with a runtime not available message (Pipenv)', LATEST_PYTHON_3_6
+    end
   end
 
   context 'with a Pipfile.lock containing python_version 3.7' do
-    let(:app) { Hatchet::Runner.new('spec/fixtures/pipenv_python_3.7') }
+    let(:allow_failure) { false }
+    let(:app) { Hatchet::Runner.new('spec/fixtures/pipenv_python_3.7', allow_failure: allow_failure) }
 
-    include_examples 'builds using Pipenv with the requested Python version', LATEST_PYTHON_3_7
+    context 'when using Heroku-18 or Heroku-20', stacks: %w[heroku-18 heroku-20] do
+      include_examples 'builds using Pipenv with the requested Python version', LATEST_PYTHON_3_7
+    end
+
+    context 'when using Heroku-22', stacks: %w[heroku-22] do
+      let(:allow_failure) { true }
+
+      # Python 3.7 is in the security fix only stage of its lifecycle, so has not been built for newer stacks.
+      include_examples 'aborts the build with a runtime not available message (Pipenv)', LATEST_PYTHON_3_7
+    end
   end
 
   context 'with a Pipfile.lock containing python_version 3.8' do
-    let(:app) { Hatchet::Runner.new('spec/fixtures/pipenv_python_3.8') }
+    let(:allow_failure) { false }
+    let(:app) { Hatchet::Runner.new('spec/fixtures/pipenv_python_3.8', allow_failure: allow_failure) }
 
-    include_examples 'builds using Pipenv with the requested Python version', LATEST_PYTHON_3_8
+    context 'when using Heroku-18 or Heroku-20', stacks: %w[heroku-18 heroku-20] do
+      include_examples 'builds using Pipenv with the requested Python version', LATEST_PYTHON_3_8
+    end
+
+    context 'when using Heroku-22', stacks: %w[heroku-22] do
+      let(:allow_failure) { true }
+
+      # Python 3.8 is in the security fix only stage of its lifecycle, so has not been built for newer stacks.
+      include_examples 'aborts the build with a runtime not available message (Pipenv)', LATEST_PYTHON_3_8
+    end
   end
 
   context 'with a Pipfile.lock containing python_version 3.9' do