bjdata can't be pip-installed on PyPI/PyPy: numpy is an unconditional build dependency even though the C extension is skipped on PyPy

[oberstet]

Summary

bjdata cannot be installed on PyPy via a normal pip install bjdata using PYBJDATA_NO_EXTENSION=1. The install fails while building the numpy build dependency from source — even though bjdata deliberately skips its C extension on PyPy and runs as pure Python there. numpy is pulled in unconditionally at build time and is, for the PyPy/pure-Python build, entirely unnecessary.

This is bjdata 0.6.6 (latest on PyPI).

What happens

On PyPy 3.11, pip install bjdata (building from the PyPI sdist, since no wheels are published) does:

Collecting bjdata
  Downloading bjdata-0.6.6.tar.gz
  Installing build dependencies ...
    Using cached oldest_supported_numpy-...
    Collecting numpy==1.23.2 (from oldest-supported-numpy)
    Downloading numpy-1.23.2.tar.gz
    Building wheel for numpy (pyproject.toml): ... error      <-- fails here
ERROR: Failed to build 'bjdata' when installing build dependencies for bjdata

numpy==1.23.2 has no PyPy 3.11 wheel, so pip tries to compile that (old)
numpy from source under PyPy, which fails. The build never even reaches
bjdata's own setup.py.

Root cause

Two unconditional references to numpy force this, both of which are dead weight for the PyPy/pure-Python build:

1. pyproject.toml — numpy is an unconditional PEP 517 build requirement:

   [build-system]
   requires = ["setuptools>=40", "wheel", "oldest-supported-numpy"]
   build-backend = "setuptools.build_meta"

   Under build isolation, pip installs oldest-supported-numpy before running
   setup.py. On PyPy that resolves to numpy==1.23.2 (no PyPy wheel) → source
   build → failure. (oldest-supported-numpy is also deprecated upstream in
   favor of building against the numpy 2.0 ABI.)

2. setup.py — numpy is imported at module top level:

   from numpy import get_include as numpy_get_include      # top of setup.py
   ...
   BUILD_EXTENSIONS = (
       "PYBJDATA_NO_EXTENSION" not in os.environ and python_implementation() != "PyPy"
   )
   ...
   ext_modules=([Extension("_bjdata", ..., include_dirs=[..., numpy_get_include()])]
                if BUILD_EXTENSIONS else [])

   BUILD_EXTENSIONS is always False on PyPy, so the _bjdata C extension
   is never built and numpy_get_include() is never used there — yet the
   top-level from numpy import get_include means setup.py cannot even be
   imported without numpy present.

So on PyPy, bjdata is pure Python and needs no compiler and no numpy — but the packaging requires numpy at build time anyway, and the only numpy it can find (oldest-supported-numpy → 1.23.2) is unbuildable on PyPy.

Proposed fix (two small changes)

1. setup.py: make the numpy import lazy / conditional on actually building
   the extension, e.g. import get_include only inside the if BUILD_EXTENSIONS
   branch instead of at module top level. Then setup.py needs numpy only when
   the C extension is actually compiled (CPython, extension enabled).

2. pyproject.toml: drop the deprecated oldest-supported-numpy and require
   a modern numpy that ships PyPy wheels, ideally gated so it is only required
   where the extension is built, e.g.:

   [build-system]
   requires = [
     "setuptools>=40",
     "wheel",
     "numpy>=2.0.0; platform_python_implementation == 'CPython'",
   ]
   build-backend = "setuptools.build_meta"

   (numpy 2.x publishes PyPy 3.11 wheels, so even an unconditional numpy>=2.0.0 would already let the build succeed on PyPy.)

With either/both, pip install bjdata would succeed on PyPy as a pure-Python package, with no numpy build.

(Optionally: publishing binary wheels for bjdata would also help — currently PyPI has sdist only, so every install builds from source.)

Impact / where we hit this

Autobahn|Python provides a WAMP "ubjson" serializer and is migrating it from the unmaintained py-ubjson to the maintained bjdata (issue crossbario/autobahn-python#1849, PR #1855). We expose bjdata via an optional autobahn[serialization] extra. Our CI's PyPy 3.11 job fails to install it for exactly the reason above. Because the failure is in
bjdata's build dependencies (before setup.py runs), setting PYBJDATA_NO_EXTENSION=1 does not help.

As a workaround on our side we are restricting the bjdata dependency to CPython, which means the UBJSON serializer is unavailable on PyPy. We'd love to relax that once bjdata installs cleanly on PyPy.

Thanks for maintaining bjdata!