-
Notifications
You must be signed in to change notification settings - Fork 16
[0.2.0][CI] Add E2E integration test pipeline #72
Copy link
Copy link
Closed
Labels
Milestone
Description
Summary
Add an end-to-end integration test pipeline that validates the full Simili Bot triage flow on every PR to main.
Problem
There was no automated way to verify that the bot correctly:
- Posts triage comments on issues
- Detects similar/duplicate issues via vector search
- Applies labels
- Does not enter infinite comment loops
Regressions in any of these areas would only be caught manually.
Solution
A GitHub Actions workflow (.github/workflows/e2e-test.yml) that:
- Creates a private test repository under the bot user account
- Seeds it with 5 representative issues and indexes them into a dedicated Qdrant collection
- Installs the PR version of Simili Bot via the generated
triage.yml - Creates a trigger issue designed to match the seed issues
- Polls for the bot's triage comment (similarity detection, duplicate warning, labels)
- Verifies no infinite loop occurs
- Cleans up the Qdrant collection (always, even on failure)
- Posts a result summary comment on the PR
Acceptance Criteria
- E2E workflow passes on every PR to
main - Bot comment is detected within 5 minutes
- Similarity/duplicate detection is verified in the comment body
- Qdrant collection is cleaned up after each run
- Test repository remains private
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Done