Skip to content

feat(helm): add configurable workspace listing limits#900

Merged
tiborsimko merged 1 commit intoreanahub:masterfrom
michaelbuchar:feature/performance-of-file-listings
Apr 22, 2026
Merged

feat(helm): add configurable workspace listing limits#900
tiborsimko merged 1 commit intoreanahub:masterfrom
michaelbuchar:feature/performance-of-file-listings

Conversation

@michaelbuchar
Copy link
Copy Markdown

@michaelbuchar michaelbuchar commented Jul 11, 2025

Add WORKSPACE_DISPLAY_FILE_LIMIT and FORCE_GARBAGE_COLLECTION
environment variables for reana-workflow-controller to the
Helm chart README.

Closes reanahub/reana-workflow-controller#644

@michaelbuchar michaelbuchar self-assigned this Jul 11, 2025
@michaelbuchar michaelbuchar moved this to In review in 0.95.0 Jul 11, 2025
@michaelbuchar michaelbuchar changed the title feat(config): limit workspace file listing and implement GC (#899) feat(config): limit workspace file listing and implement GC Jul 11, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Jul 11, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 29.86%. Comparing base (301fc2a) to head (cdcaa96).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #900   +/-   ##
=======================================
  Coverage   29.86%   29.86%           
=======================================
  Files          26       26           
  Lines        2823     2823           
=======================================
  Hits          843      843           
  Misses       1980     1980           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Jul 11, 2025
…#900)

Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 2dcf76e to 736ed9f Compare July 11, 2025 14:28
@michaelbuchar michaelbuchar marked this pull request as ready for review July 11, 2025 14:37
@michaelbuchar michaelbuchar requested a review from tiborsimko July 11, 2025 14:38
@michaelbuchar michaelbuchar changed the title feat(config): limit workspace file listing and implement GC feat(helm): configure workspace file limit and GC Dec 4, 2025
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Dec 4, 2025
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 736ed9f to 56ffec7 Compare December 4, 2025 09:06
@michaelbuchar michaelbuchar removed this from 0.95.0 Dec 4, 2025
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Jan 28, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 56ffec7 to b7a3759 Compare January 28, 2026 08:49
Comment thread helm/reana/values.yaml Outdated
REANA_OPENSEARCH_USE_SSL: true
REANA_OPENSEARCH_CA_CERTS: "/code/certs/ca.crt"
REANA_OPENSEARCH_USER: reana
WORKSPACE_DISPLAY_FILE_LIMIT: 100000
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we document the new environment variables in README.md as part of this PR? Or shall we do it later as part of #901?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do it now so that we don't forget about it :)

michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Feb 6, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from b7a3759 to 2f3ec4c Compare February 6, 2026 10:35
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Feb 6, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 2f3ec4c to 4a67098 Compare February 6, 2026 10:37
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Feb 6, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 4a67098 to df9a8b5 Compare February 6, 2026 10:37
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Feb 6, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from df9a8b5 to 6c2c01d Compare February 6, 2026 10:38
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Feb 6, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 401fcdd to 6cd96cc Compare February 6, 2026 10:41
@michaelbuchar michaelbuchar deleted the feature/performance-of-file-listings branch April 21, 2026 07:40
@michaelbuchar michaelbuchar restored the feature/performance-of-file-listings branch April 21, 2026 07:42
@michaelbuchar michaelbuchar reopened this Apr 21, 2026
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Apr 21, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 6cd96cc to df9813c Compare April 21, 2026 07:48
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Apr 21, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from df9813c to f38e835 Compare April 21, 2026 07:49
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Apr 22, 2026
Set default values for FORCE_GARBAGE_COLLECTION and
WORKSPACE_DISPLAY_FILE_LIMIT environment variables in Helm.
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from f38e835 to 35b905d Compare April 22, 2026 08:49
michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Apr 22, 2026
Set default values for WORKSPACE_DISPLAY_FILE_LIMIT and
FORCE_GARBAGE_COLLECTION in the Helm chart for
reana-workflow-controller.

Closes reanahub/reana-workflow-controller#644
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 35b905d to ee64327 Compare April 22, 2026 09:45
@michaelbuchar michaelbuchar changed the title feat(helm): configure workspace file limit and GC feat(helm): add workspace listing defaults Apr 22, 2026
Comment thread helm/reana/values.yaml Outdated
REANA_OPENSEARCH_CA_CERTS: "/code/certs/ca.crt"
REANA_OPENSEARCH_USER: reana
WORKSPACE_DISPLAY_FILE_LIMIT: 100000
FORCE_GARBAGE_COLLECTION: ""
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FORCE_GARBAGE_COLLECTION can be removed from values.yaml since reana-workflow-controller already defaults to "" when the env var is not set:

_gc_env = os.getenv("FORCE_GARBAGE_COLLECTION", "")

The README documentation added in this PR is sufficient for discoverability.

(Same applies to REANA_QUOTA_MANAGEMENT_SECRET which could be cleaned up separately — reana-server already defaults to "" in its config.py.)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Comment thread helm/reana/values.yaml Outdated
REANA_OPENSEARCH_USE_SSL: true
REANA_OPENSEARCH_CA_CERTS: "/code/certs/ca.crt"
REANA_OPENSEARCH_USER: reana
WORKSPACE_DISPLAY_FILE_LIMIT: 100000
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for WORKSPACE_DISPLAY_FILE_LIMITreana-workflow-controller already defaults to 100000 in its config.py:

WORKSPACE_DISPLAY_FILE_LIMIT = int(os.getenv("WORKSPACE_DISPLAY_FILE_LIMIT", "100000"))

So this line can be removed from values.yaml as well, keeping only the README documentation.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@tiborsimko
Copy link
Copy Markdown
Member

tiborsimko commented Apr 22, 2026

Nit: consider rewording the commit message for release notes readability. The current headline "add workspace listing defaults" is a bit vague. Suggestion:

feat(helm): add configurable workspace listing limits (#900)

Add WORKSPACE_DISPLAY_FILE_LIMIT and FORCE_GARBAGE_COLLECTION
environment variables for reana-workflow-controller to the
Helm chart README.

Closes reanahub/reana-workflow-controller#644

michaelbuchar pushed a commit to michaelbuchar/reana that referenced this pull request Apr 22, 2026
Add WORKSPACE_DISPLAY_FILE_LIMIT and FORCE_GARBAGE_COLLECTION
environment variables for reana-workflow-controller to the
Helm chart README.

Closes reanahub/reana-workflow-controller#644
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from ee64327 to 1694191 Compare April 22, 2026 13:20
Add WORKSPACE_DISPLAY_FILE_LIMIT and FORCE_GARBAGE_COLLECTION
environment variables for reana-workflow-controller to the
Helm chart README.

Closes reanahub/reana-workflow-controller#644
@michaelbuchar michaelbuchar force-pushed the feature/performance-of-file-listings branch from 1694191 to cdcaa96 Compare April 22, 2026 13:21
@michaelbuchar michaelbuchar changed the title feat(helm): add workspace listing defaults feat(helm): add configurable workspace listing limits Apr 22, 2026
@tiborsimko tiborsimko merged commit cdcaa96 into reanahub:master Apr 22, 2026
17 checks passed
@michaelbuchar michaelbuchar deleted the feature/performance-of-file-listings branch April 22, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

workspace: ensure smooth performance of file listings for workspaces containing 250k files

2 participants