Skip to content

WIP: Add server login integration tests#5311

Open
manuroe wants to merge 6 commits intodevelopfrom
feat/server-login-integration-tests
Open

WIP: Add server login integration tests#5311
manuroe wants to merge 6 commits intodevelopfrom
feat/server-login-integration-tests

Conversation

@manuroe
Copy link
Copy Markdown
Member

@manuroe manuroe commented Mar 27, 2026

Adds a ServerLoginTests class with a single testLogin method that verifies login works against a configurable server.

The login helper auto-detects whether the server uses a Keycloak identity provider (via the "Continue with Keycloak" link) and adapts the web UI interaction accordingly, so the same test works against any OIDC server. It also handles stale MAS sessions by looking for "Sign out" or "Use another account" buttons before attempting login.

A new server-login-tests CLI command runs only this test class. A GitHub Actions matrix job in the integration tests workflow feeds server-specific secrets as the standard INTEGRATION_TESTS_HOST/USERNAME/PASSWORD variables. The matrix currently covers three servers: default, lts, lts-1.

@manuroe manuroe requested a review from a team as a code owner March 27, 2026 13:40
@manuroe manuroe requested review from pixlwave and removed request for a team March 27, 2026 13:40
- name: Run tests
run: swift run -q tools ci server-login-tests
env:
INTEGRATION_TESTS_HOST: ${{ secrets[matrix.host_secret] }}

Check warning

Code scanning / zizmor

excessively provisioned secrets Warning

excessively provisioned secrets
run: swift run -q tools ci server-login-tests
env:
INTEGRATION_TESTS_HOST: ${{ secrets[matrix.host_secret] }}
INTEGRATION_TESTS_USERNAME: ${{ secrets[matrix.username_secret] }}

Check warning

Code scanning / zizmor

excessively provisioned secrets Warning

excessively provisioned secrets
env:
INTEGRATION_TESTS_HOST: ${{ secrets[matrix.host_secret] }}
INTEGRATION_TESTS_USERNAME: ${{ secrets[matrix.username_secret] }}
INTEGRATION_TESTS_PASSWORD: ${{ secrets[matrix.password_secret] }}

Check warning

Code scanning / zizmor

excessively provisioned secrets Warning

excessively provisioned secrets
@manuroe manuroe force-pushed the feat/server-login-integration-tests branch from 1f366a7 to 029c41b Compare March 27, 2026 13:52
@manuroe
Copy link
Copy Markdown
Member Author

manuroe commented Mar 27, 2026

No luck with the first attempt :D

image

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 27, 2026

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
911 2 909 0
View the top 1 failed test(s) by shortest run time
ServerLoginTests::testLogin()
Stack Traces | 562s run time
XCTAssertNotNil failed - INTEGRATION_TESTS_HOST environment variable must be set for this test to run. (ServerLoginTests.swift:19)
View the full list of 1 ❄️ flaky test(s)
UserFlowTests::testUserFlow()

Flake rate in main: 64.63% (Passed 162 times, Failed 296 times)

Stack Traces | 389s run time
XCTAssertTrue failed (Common.swift:140)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@manuroe manuroe changed the title Add server login integration tests WIP: Add server login integration tests Mar 30, 2026
manuroe added 3 commits March 31, 2026 17:56
Adds a generic ServerLoginTests class with a single testLogin method
that reads the server host, username and password from environment
variables. The login helper auto-detects whether the server uses a
Keycloak identity provider and adapts the web UI interaction accordingly,
so the same test works against any OIDC server.

A new server-login-tests CLI command runs only this test class. A GitHub
Actions matrix job feeds server-specific secrets as the standard
INTEGRATION_TESTS_HOST/USERNAME/PASSWORD variables, making it trivial
to extend coverage to additional servers by adding new matrix entries.
@manuroe manuroe force-pushed the feat/server-login-integration-tests branch from 0d9e3b9 to 42288a1 Compare March 31, 2026 16:04
@manuroe manuroe force-pushed the feat/server-login-integration-tests branch from de24ea3 to a98d41d Compare April 3, 2026 13:26
@manuroe manuroe had a problem deploying to integration-tests April 3, 2026 13:39 — with GitHub Actions Failure
@manuroe manuroe had a problem deploying to integration-tests April 3, 2026 13:39 — with GitHub Actions Failure
@manuroe manuroe had a problem deploying to integration-tests April 3, 2026 13:39 — with GitHub Actions Failure
@manuroe manuroe had a problem deploying to integration-tests April 3, 2026 15:15 — with GitHub Actions Failure
@manuroe manuroe had a problem deploying to integration-tests April 3, 2026 15:15 — with GitHub Actions Failure
@manuroe manuroe had a problem deploying to integration-tests April 3, 2026 15:15 — with GitHub Actions Failure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants