build(python): upgrade to Snakemake 9.16.3 #514
Merged
tiborsimko merged 2 commits intoreanahub:masterfrom Mar 3, 2026
Merged
Conversation
tiborsimko
reviewed
Mar 2, 2026
| "snakemake==7.32.4 ; python_version<'3.11'", | ||
| "pulp>=2.7.0,<2.8.0 ; python_version<'3.11'", | ||
| "snakemake==8.27.1 ; python_version>='3.11'", | ||
| "snakemake==9.16.3 ; python_version>='3.11'", |
Member
There was a problem hiding this comment.
Thanks for starting this up! I verified that most examples work nicely, however there were changes in the logging infrastructure so we need to take care of updating reana-workflow-engine-snakemake accordingly. I'll do that and amend this PR...
tiborsimko
force-pushed
the
feat/upgrade-snakemake-v9
branch
from
4bdefe5 to
fbea72d
Compare
tiborsimko
pushed a commit
to alefisico/reana-commons
that referenced
this pull request
Mar 2, 2026
Upgrade Snakemake to 9.16.3. Add plugin `snakemake-executor-plugin-kubernetes` to the list of dependencies. This is now necessary since upstream Snakemake extracted executors to plugins. Upgrade `snakemake-storage-plugin-xrootd` plugin to 1.0.0.
tiborsimko
force-pushed
the
feat/upgrade-snakemake-v9
branch
from
fbea72d to
7895ee8
Compare
tiborsimko
reviewed
Mar 2, 2026
| - [Adelina Lintuluoto](https://orcid.org/0000-0002-0726-1452) | ||
| - [Agisilaos Kounelis](https://orcid.org/0000-0001-9312-3189) | ||
| - [Alastair Lyall](https://orcid.org/0009-0000-4955-8935) | ||
| - [Alejandro Gomez Espinosa](https://orcid.org/0000-0002-9443-7769) |
Member
There was a problem hiding this comment.
@alefisico Thanks, I have slightly amended this PR and also added you to the list of contributors.
tiborsimko
added a commit
to alefisico/reana-commons
that referenced
this pull request
Mar 2, 2026
Snakemake 9.16.3 requires snakemake-executor-plugin-kubernetes>=0.1.5 on Python 3.11+, which in turn needs kubernetes>=27.2.0. Since kubernetes 27+ requires oauthlib>=3.2.2 which is incompatible with reana-server invenio packages (flask-oauthlib needs oauthlib<3.0.0), the executor plugin is split into a separate "snakemake-kubernetes" extra. The "kubernetes" extra upper pin is kept at <27.0.0. Co-authored-by: Tibor Šimko <tibor.simko@cern.ch>
tiborsimko
force-pushed
the
feat/upgrade-snakemake-v9
branch
from
7895ee8 to
c2dc33d
Compare
tiborsimko
changed the title
tiborsimko
reviewed
Mar 2, 2026
| ], | ||
| "kubernetes": [ | ||
| "kubernetes>=22.0.0,<23.0.0", | ||
| "kubernetes>=22.0.0,<27.0.0", # upper pin due to oauthlib 3.x incompatibility with reana-server invenio packages |
Member
There was a problem hiding this comment.
FYI this resulted from more local testing. And we need to merge reanahub/pytest-reana#149 first and release the new pytest-reana on PyPI in order to solve failing CI tests.
Member
There was a problem hiding this comment.
Here's the set of related PRs, for reference:
- build(python): remove kubernetes extra from reana-commons pytest-reana#149
- test(snakemake): adapt engine logs tests to Snakemake 9 reana-demo-helloworld#75
- test(snakemake): adapt engine logs tests to Snakemake 9 reana-demo-root6-roofit#80 test(snakemake): adapt engine logs tests to Snakemake 9 reana-demo-cms-h4l#39
- test(snakemake): adapt disk size and log tests to Snakemake 9 reana-demo-worldpopulation#68
- feat(runner): upgrade to Snakemake v9 reana-workflow-engine-snakemake#121
tiborsimko
added a commit
to alefisico/reana-commons
that referenced
this pull request
Mar 3, 2026
Snakemake 9.16.3 requires snakemake-executor-plugin-kubernetes>=0.1.5 on Python 3.11+, which in turn needs kubernetes>=27.2.0. Since kubernetes 27+ requires oauthlib>=3.2.2 which is incompatible with reana-server invenio packages (flask-oauthlib needs oauthlib<3.0.0), the executor plugin is split into a separate "snakemake-kubernetes" extra. The "kubernetes" extra upper pin is kept at <27.0.0. Co-authored-by: Tibor Šimko <tibor.simko@cern.ch>
tiborsimko
force-pushed
the
feat/upgrade-snakemake-v9
branch
from
c2dc33d to
93ff450
Compare
tiborsimko
added a commit
to alefisico/reana-commons
that referenced
this pull request
Mar 3, 2026
Snakemake 9.16.3 requires snakemake-executor-plugin-kubernetes>=0.1.5 on Python 3.11+, which in turn needs kubernetes>=27.2.0. Since kubernetes 27+ requires oauthlib>=3.2.2 which is incompatible with reana-server invenio packages (flask-oauthlib needs oauthlib<3.0.0), the executor plugin is split into a separate "snakemake-kubernetes" extra. The "kubernetes" extra upper pin is kept at <27.0.0. Co-authored-by: Tibor Šimko <tibor.simko@cern.ch>
tiborsimko
force-pushed
the
feat/upgrade-snakemake-v9
branch
from
93ff450 to
bc7cd0f
Compare
Snakemake 9.16.3 requires snakemake-executor-plugin-kubernetes>=0.1.5 on Python 3.11+, which in turn needs kubernetes>=27.2.0. Since kubernetes 27+ requires oauthlib>=3.2.2 which is incompatible with reana-server invenio packages (flask-oauthlib needs oauthlib<3.0.0), the executor plugin is split into a separate "snakemake-kubernetes" extra. The "kubernetes" extra upper pin is kept at <27.0.0. Since the "kubernetes" and "snakemake-kubernetes" extras conflict and cannot be installed together, the tox test configuration and the CI workflow now list compatible extras explicitly instead of using "all". Co-authored-by: Tibor Šimko <tibor.simko@cern.ch>
tiborsimko
force-pushed
the
feat/upgrade-snakemake-v9
branch
from
bc7cd0f to
621d52e
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #514 +/- ##
=======================================
Coverage 41.14% 41.14%
=======================================
Files 28 28
Lines 2066 2066
=======================================
Hits 850 850
Misses 1216 1216
🚀 New features to boost your workflow:
|
tiborsimko
force-pushed
the
feat/upgrade-snakemake-v9
branch
from
e5b7751 to
5e92a2f
Compare
tiborsimko
approved these changes
Mar 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR updates the Snakemake dependency to v9.16.3 for Python 3.11+ environments.
Changes:
snakemakepin to9.16.3.snakemake-executor-plugin-kubernetes(required since Snakemake v8+ extracted executors to plugins).Verification:
reana-workflow-engine-snakemakewith this branch and running theroot6-roofitdemo successfully.