Skip to content

[0.2.0][CI] Add E2E integration test pipeline #72

@Kavirubc

Description

@Kavirubc

Summary

Add an end-to-end integration test pipeline that validates the full Simili Bot triage flow on every PR to main.

Problem

There was no automated way to verify that the bot correctly:

  • Posts triage comments on issues
  • Detects similar/duplicate issues via vector search
  • Applies labels
  • Does not enter infinite comment loops

Regressions in any of these areas would only be caught manually.

Solution

A GitHub Actions workflow (.github/workflows/e2e-test.yml) that:

  1. Creates a private test repository under the bot user account
  2. Seeds it with 5 representative issues and indexes them into a dedicated Qdrant collection
  3. Installs the PR version of Simili Bot via the generated triage.yml
  4. Creates a trigger issue designed to match the seed issues
  5. Polls for the bot's triage comment (similarity detection, duplicate warning, labels)
  6. Verifies no infinite loop occurs
  7. Cleans up the Qdrant collection (always, even on failure)
  8. Posts a result summary comment on the PR

Acceptance Criteria

  • E2E workflow passes on every PR to main
  • Bot comment is detected within 5 minutes
  • Similarity/duplicate detection is verified in the comment body
  • Qdrant collection is cleaned up after each run
  • Test repository remains private

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Done

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions