bug(calm-studio): E2E tests job fails on main with 'playwright: not found'

[rocketstack-matt]

Bug Report

The E2E Tests job in the Build CALM Studio workflow has been failing on every push to main since the calm-suite workflow was introduced in #2329. The job fails at the Install Playwright browsers step with sh: 1: playwright: not found (exit code 127), so no calm-studio E2E tests have ever actually run on main.

Because the job is currently gated with if: github.event_name == 'push' && github.ref == 'refs/heads/main' (see build-calm-studio.yml L65), the failure is invisible on PRs and was not caught at review time.

Latest failing run: https://github.com/finos/architecture-as-code/actions/runs/24829083294 (merge of #2371 — the PR is not the cause, just the trigger; the same failure has occurred on every main push since #2329 merged on 2026-04-11).

Steps to Reproduce:

1. Push any change under calm-suite/calm-studio/** to main (or re-run the E2E Tests job on any recent main build).
2. Observe the Install Playwright browsers step.
3. The step runs npx playwright install --with-deps chromium from the calm-suite/calm-studio working directory.

Expected Result:

Playwright's chromium browser installs successfully and the E2E tests run.

Actual Result:

Run npx playwright install --with-deps chromium
sh: 1: playwright: not found
##[error]Process completed with exit code 127.

The step fails because playwright / @playwright/test is declared only in the apps/studio workspace (calm-suite/calm-studio/apps/studio/package.json), not at the root of calm-suite/calm-studio. Under pnpm workspaces, the playwright CLI lives in apps/studio/node_modules/.bin/, not in the root node_modules/.bin/, so npx playwright from the calm-suite/calm-studio directory cannot find the binary.

Environment:

• Workflow: .github/workflows/build-calm-studio.yml
• Failing step: Install Playwright browsers (L86-87)
• Runner: ubuntu-latest
• Package manager: pnpm 9 workspaces
• First failing run: https://github.com/finos/architecture-as-code/actions/runs/24280610975 (merge of feat(calm-suite): add CALM Guard and CALM Studio projects #2329, 2026-04-11)
• Most recent failing run: https://github.com/finos/architecture-as-code/actions/runs/24829083294 (merge of test(calm-core): improve validation branch coverage from 77% to 98% #2371, 2026-04-23)

Additional Context:

Proposed fix — run the install via the workspace where Playwright is actually a dependency, e.g.:

- name: Install Playwright browsers
  run: pnpm --filter @calmstudio/studio exec playwright install --with-deps chromium

Also, the Build step currently runs after the Playwright install; the E2E tests likely also need the workspace to be built before test:e2e — worth verifying the step order once the install is fixed.

Also proposing: run the E2E job on pull requests, not only on push to main. The current gating (L65) hid this regression for ~2 weeks across ~8 merges. Enabling it on PRs — the existing paths: trigger (calm-suite/calm-studio/**, .github/workflows/build-calm-studio.yml) already keeps CI cost narrow — would ensure E2E failures are caught at review time rather than after merge.

Concretely, drop the push-only gate:

e2e-tests:
    name: E2E Tests
    runs-on: ubuntu-latest
    needs: build-lint-test
    # remove: if: github.event_name == 'push' && github.ref == 'refs/heads/main'

Tasks

• Fix the Install Playwright browsers step so it resolves the playwright CLI from the @calmstudio/studio workspace.
• Verify the full E2E job runs green on main end-to-end.
• Remove the push-only gate so E2E Tests run on pull requests, preventing future blind spots.