Dependency upgrade/cleanup after dropping py3.9 by agoscinski · Pull Request #7245 · aiidateam/aiida-core

agoscinski · 2026-03-01T17:36:13Z

Removing the two dependencies importlib-metadata and get-annotations that are not anymore needed since we dropped py3.9, and update asyncssh to a version that supports py3.9.

We could also replace pytz with zoneinfo but that affects the hash function because the binary representation is not the same. A database migration seems a bit overkill to drop one dependency.

codecov · 2026-03-01T20:08:02Z

Codecov Report

❌ Patch coverage is 83.33333% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.86%. Comparing base (b289cfe) to head (7b3d00b).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/aiida/cmdline/params/types/identifier.py	50.00%	1 Missing ⚠️
src/aiida/cmdline/params/types/plugin.py	75.00%	1 Missing ⚠️
src/aiida/manage/tests/pytest_fixtures.py	50.00%	1 Missing ⚠️
src/aiida/plugins/entry_point.py	83.34%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7245      +/-   ##
==========================================
+ Coverage   79.85%   79.86%   +0.01%     
==========================================
  Files         566      566              
  Lines       43963    43958       -5     
==========================================
  Hits        35103    35103              
+ Misses       8860     8855       -5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

agoscinski · 2026-03-02T10:24:07Z

Consider also this #7240 (comment)
EDIT: Considered but typing changes will be done in a later PR shortly before next minor release

agoscinski · 2026-03-02T10:32:15Z

Consider #7240 (comment)
EDIT: Considered the asyncssh upgrade is in the PR

agoscinski · 2026-03-09T19:51:09Z

Consider upgrading pytest_asyncio https://github.com/aiidateam/aiida-core/pull/7258/changes#r2889517642

danielhollas · 2026-03-17T11:44:03Z

Consider upgrading pytest_asyncio https://github.com/aiidateam/aiida-core/pull/7258/changes#r2889517642

Let's do that in a separate PR, last time I tried it I ran into some ugly issues.

danielhollas

Great cleanup!

danielhollas · 2026-04-14T16:14:34Z

+UnionType = types.UnionType

-# Fallback for Python 3.9 and older
-from typing_extensions import ParamSpec


One more thing like this: Move TypeAlias import from typing_extensions to typing in aiida/tools/graph/age_entities.py

So I only changed it in src/aiida/engine/processes/functions.py and not in the other file. There will be a larger PR where we will use pyupgrade to automatically update every source file to 3.10 syntax and this will include also changes as removing typing_extensions imports if it is not done automatically by the tool. We want to do this right before the next minor release because it would create a lot of annoying diff noise resulting in a loft of merge conflicts for the patch releases of 2.8.x that still supports 3.9. In src/aiida/engine/processes/functions.py we already changed this part of the code so it will probably anyway result in a merge conflict. Hope this is okay.

Sure, fair enough!

`get-annotations` was a backport needed for Python <3.10. Use `inspect.get_annotations` from the stdlib instead. Run pre-commit to auto-fix TOML formatting in uv.lock and regenerate environment.yml after dependency changes. Co-authored-by: Daniel Hollas <danekhollas@gmail.com>

Replace `importlib_metadata` backport with `importlib.metadata` from the standard library, which is sufficient for the supported Python versions (>=3.10). Remove now-unnecessary `# type: ignore[no-untyped-call]` comments on `EntryPoint.load()` calls since the stdlib version is fully typed. Fix `eps()` to flatten `SelectableGroups` (dict of group name to `EntryPoints`) returned by `entry_points()` into a flat sorted `EntryPoints` list, since the stdlib `entry_points()` returns a dict-like `SelectableGroups` rather than a flat iterable. Fix `EntryPoints` construction in test fixtures to use list unpacking instead of tuple concatenation, resolving mypy overload resolution errors.

Regenerate uv.lock and environment.yml via pre-commit hooks after bumping asyncssh version constraint.

`get-annotations` was a backport needed for Python <3.10. Use `inspect.get_annotations` from the stdlib instead. Co-authored-by: Daniel Hollas <danekhollas@gmail.com>

Replace `importlib_metadata` backport with `importlib.metadata` from the standard library, which is sufficient for the supported Python versions (>=3.10). Remove now-unnecessary `# type: ignore[no-untyped-call]` comments on `EntryPoint.load()` calls since the stdlib version is fully typed. Flatten `SelectableGroups` returned by `entry_points()` on Python <3.12 via `dict.values()` to get a flat list of `EntryPoint` objects, since `SelectableGroups` iterates over group name strings, not entry points. Fix `EntryPoints` construction in test fixtures to use list unpacking instead of tuple concatenation, resolving mypy overload resolution errors.

Regenerate uv.lock and environment.yml via pre-commit hooks after bumping asyncssh version constraint.

agoscinski · 2026-04-14T20:18:25Z

-    all_eps = entry_points()
-    return EntryPoints(sorted(all_eps, key=lambda x: x.group))
+    eps = entry_points()
+    # Python <3.12 returns SelectableGroups (dict subclass); flatten it.


We have to sadly adapt it
https://docs.python.org/3.12/library/importlib.metadata.html

importlib_metadata 5.0 and Python 3.12, entry_points always returns an EntryPoints object.

But I think its worth removing a dependency

`get-annotations` was a backport needed for Python <3.10. Use `inspect.get_annotations` from the stdlib instead. Co-authored-by: Daniel Hollas <danekhollas@gmail.com>

Replace `importlib_metadata` backport with `importlib.metadata` from the standard library, which is sufficient for the supported Python versions (>=3.10). Handle the `entry_points()` return type difference across Python versions using `sys.version_info` guard: Python <3.12 returns SelectableGroups (dict subclass iterating group names), while >=3.12 returns EntryPoints directly. Remove now-unnecessary `# type: ignore[no-untyped-call]` comments on `EntryPoint.load()` calls since the stdlib version is fully typed. Fix `EntryPoints` construction in test fixtures to use list unpacking instead of tuple concatenation, resolving mypy overload resolution errors.

Regenerate uv.lock and environment.yml via pre-commit hooks after bumping asyncssh version constraint.

agoscinski · 2026-04-14T20:53:04Z

+    else:
+        # Python <3.12: entry_points() returns SelectableGroups (dict subclass)
+        # that iterates group names, not EntryPoints. Flatten via .values().
+        all_eps = [ep for group_eps in entry_points().values() for ep in group_eps]


Had to iterate through this logic multiple times. I think this is the clearest way. Its a bit annoying but I think its worse the dependency drop.

Ugh, this is such a mess, how can they be changing the interface in every other python version?!? Agreed that still worth dropping the dependency.

agoscinski · 2026-04-14T20:56:44Z

We might just keep this PR stale until we plan the minor release. After going through all changes I feel like the changes are not really needed for any subsequent changes so there is no rush for merging this and it will make our life just harder for patch releases.

danielhollas · 2026-04-14T21:43:03Z

We might just keep this PR stale until we plan the minor release. After going through all changes I feel like the changes are not really needed for any subsequent changes so there is no rush for merging this and it will make our life just harder for patch releases.

Agreed on principle, but at the same time, these changes seem quite localized, and seem unlikely to induce conflicts with other PRs? So I'd be for merging this.

If you plan to leave this stale, than I'd propose at the very least merge a separate PR with the uv.lock change.

agoscinski · 2026-04-15T06:03:56Z

Agreed on principle, but at the same time, these changes seem quite localized, and seem unlikely to induce conflicts with other PRs? So I'd be for merging this.

Also true, but I thought that the uv changes might be actually annoying to merge for patch releases, on the other hand these conflicts can be fixed by the pre-commit hook. And touching a PR after some months is also often almost like starting from scratch. I don't have a strong opinion on this, we can also merge now. Your call.

danielhollas · 2026-04-15T12:06:34Z

but I thought that the uv changes might be actually annoying to merge for patch releases,

Yeah, but that would mean you'd have to block all PRs that touch dependencies. And as you say, conflicts in uv.lock can be generally solved by regenerating the uv.lock (which shouldn't really matter in terms of patch version release, since the uv.lock only influences what gets tested in CI.

I'd say on the opposite side, the uv.lock should be merged especially, since otherwise every PR that makes a small change in dependency might get this big diff.

Also, merging only just before the minor version release is riskier since the change have not been tested on main for a while.

So overall I feel this should be merged now.

`get-annotations` was a backport needed for Python <3.10. Use `inspect.get_annotations` from the stdlib instead. Co-authored-by: Daniel Hollas <danekhollas@gmail.com>

Replace `importlib_metadata` backport with `importlib.metadata` from the standard library, which is sufficient for the supported Python versions (>=3.10). Handle the `entry_points()` return type difference across Python versions using `sys.version_info` guard: Python <3.12 returns SelectableGroups (dict subclass iterating group names), while >=3.12 returns EntryPoints directly. Remove now-unnecessary `# type: ignore[no-untyped-call]` comments on `EntryPoint.load()` calls since the stdlib version is fully typed. Fix `EntryPoints` construction in test fixtures to use list unpacking instead of tuple concatenation, resolving mypy overload resolution errors.

GeigerJ2 · 2026-04-16T08:51:34Z

Thanks a lot, @agoscinski! Great work! Sry was a bit too late on that one ^^

agoscinski force-pushed the py3.9-post-cleanup branch 2 times, most recently from 156e311 to ab9c709 Compare March 1, 2026 19:42

agoscinski force-pushed the py3.9-post-cleanup branch from 728a4c5 to 07373ba Compare March 4, 2026 14:26

agoscinski changed the title ~~Dependency cleanup after dropping py3.9~~ Dependency upgrade/cleanup after dropping py3.9 Mar 4, 2026

danielhollas approved these changes Mar 17, 2026

View reviewed changes

Comment thread src/aiida/engine/processes/functions.py Outdated

Comment thread src/aiida/engine/processes/functions.py Outdated

danielhollas reviewed Apr 14, 2026

View reviewed changes

agoscinski changed the base branch from drop-py3.9 to main April 14, 2026 18:49

agoscinski added a commit to agoscinski/aiida-core that referenced this pull request Apr 14, 2026

⬆️ Bump asyncssh to ~=2.22.0 (aiidateam#7245)

d3a4e57

Regenerate uv.lock and environment.yml via pre-commit hooks after bumping asyncssh version constraint.

agoscinski force-pushed the py3.9-post-cleanup branch from 07373ba to d3a4e57 Compare April 14, 2026 19:45

agoscinski added a commit to agoscinski/aiida-core that referenced this pull request Apr 14, 2026

⬆️ Bump asyncssh to ~=2.22.0 (aiidateam#7245)

d2a495d

Regenerate uv.lock and environment.yml via pre-commit hooks after bumping asyncssh version constraint.

agoscinski force-pushed the py3.9-post-cleanup branch from d3a4e57 to d2a495d Compare April 14, 2026 19:59

agoscinski marked this pull request as ready for review April 14, 2026 20:05

agoscinski requested a review from GeigerJ2 as a code owner April 14, 2026 20:06

agoscinski commented Apr 14, 2026

View reviewed changes

agoscinski and others added 3 commits April 14, 2026 22:33

♻️ Remove get-annotations backport (aiidateam#7245)

411e906

`get-annotations` was a backport needed for Python <3.10. Use `inspect.get_annotations` from the stdlib instead. Co-authored-by: Daniel Hollas <danekhollas@gmail.com>

⬆️ Bump asyncssh to ~=2.22.0 (aiidateam#7245)

7b3d00b

Regenerate uv.lock and environment.yml via pre-commit hooks after bumping asyncssh version constraint.

agoscinski force-pushed the py3.9-post-cleanup branch from d2a495d to 7b3d00b Compare April 14, 2026 20:52

agoscinski commented Apr 14, 2026

View reviewed changes

agoscinski merged commit 36aaf8c into aiidateam:main Apr 15, 2026
31 checks passed

agoscinski deleted the py3.9-post-cleanup branch April 15, 2026 18:08

Conversation

agoscinski commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

agoscinski commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agoscinski commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agoscinski commented Mar 9, 2026

Uh oh!

danielhollas commented Mar 17, 2026

Uh oh!

danielhollas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

danielhollas Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

agoscinski Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

danielhollas Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

agoscinski Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

agoscinski Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danielhollas Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

agoscinski commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danielhollas commented Apr 14, 2026

Uh oh!

agoscinski commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danielhollas commented Apr 15, 2026

Uh oh!

Uh oh!

GeigerJ2 commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

agoscinski commented Mar 1, 2026 •

edited

Loading

codecov Bot commented Mar 1, 2026 •

edited

Loading

agoscinski commented Mar 2, 2026 •

edited

Loading

agoscinski commented Mar 2, 2026 •

edited

Loading

agoscinski Apr 14, 2026 •

edited

Loading

agoscinski commented Apr 14, 2026 •

edited

Loading

agoscinski commented Apr 15, 2026 •

edited

Loading

GeigerJ2 commented Apr 16, 2026 •

edited

Loading