Skip to content

Commit eb841f5

Browse files
chore: repo cleanup, stale doc fixes, CHANGELOG backfill
Repo hygiene - Delete stale tracked artifacts: nupkgs/ (old v1.14/v1.15 packages that pre-date the *.nupkg gitignore rule), coverage/ (Mar 15 cobertura reports), docs/PROJECT-KNOWLEDGE.md (stale shadow of root version that diverged weeks ago — root is canonical) - Add coverage/ to .gitignore so it stops getting committed AGENTS.md - Replaced with a 5-line stub pointing to CLAUDE.md (the auto-maintained agent context). Old AGENTS.md had multiple stale references: Spectra.GitHub project (doesn't exist), Spectra.Tests (split into 3 test projects long ago), spectra.profile.md at repo root (replaced by profiles/_default.yaml), and a "Recent Changes" section that ended at spec 010 (24+ specs ago). Keeps the agents.md filename convention while routing readers to the live source of truth. architecture-v5.md - Added a status header clarifying it's the original design spec, not the current state. Points readers at CLAUDE.md, PROJECT-KNOWLEDGE.md, and specs/ for current architecture. CONTRIBUTING.md - Removed reference to non-existent spec-kit/adr/ directory; replaced with the spec-kit workflow (specs/NNN-feature/) that actually exists. CHANGELOG.md - Backfilled the missing 1.36.0 entry covering all three features released in this version: spec 030 (customizable prompt templates, 11th SKILL), spec 031 (visible default profile & customization guide), spec 032 (quickstart SKILL & USAGE.md offline guide). Also notes the three Dependabot bumps merged today. Documentation accuracy fixes - README.md: 'requirements'→'acceptance criteria' in coverage section, '10 SKILLs'→'12 SKILLs', extracts 'requirements'→ 'acceptance criteria' - docs/test-format.md: requirements field→criteria field throughout the example, the fields table, the coverage mapping, and the _index.json example - docs/configuration.md: removed legacy requirements_file column; scan example uses criteria_file - docs/architecture/overview.md: 'Requirements'→'Acceptance Criteria' in the three coverage dimensions list - docs/cli-reference.md: spectra init now reports the actual artifacts it creates (12 SKILLs, profiles, prompts, CUSTOMIZATION.md, USAGE.md); removed the 'spectra mcp start' subcommand reference (the command doesn't exist — MCP is the separate spectra-mcp global tool) - docs/skills-integration.md: '9 SKILL files'→'12 SKILL files', added rows for spectra-update, spectra-prompts, spectra-quickstart - docs/execution-agent/overview.md, copilot-cli.md, generic-mcp.md, claude.md: replaced all 'spectra mcp start' / 'spectra mcp call' references with the correct spectra-mcp standalone tool invocation and JSON-RPC stdio examples; copilot-cli.md no longer points at a nonexistent .github/skills/spectra-execution/SKILL.md (the agent prompt lives in .github/agents/spectra-execution.agent.md)
1 parent 6a97159 commit eb841f5

File tree

171 files changed

+87
-51479
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

171 files changed

+87
-51479
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ packages/
2727

2828
# Test results
2929
TestResults/
30+
coverage/
3031

3132
# SPECTRA lock files
3233
.spectra.lock

AGENTS.md

Lines changed: 18 additions & 108 deletions
Original file line numberDiff line numberDiff line change
@@ -1,116 +1,26 @@
11
# AGENTS.md
22

3-
## Project Overview
3+
> **This file exists for the [agents.md](https://agents.md) convention** so external coding agents can find a project guide by name. The authoritative agent context for SPECTRA is **[`CLAUDE.md`](CLAUDE.md)** — it is auto-maintained by the spec-kit workflow and reflects the current technology stack, project structure, recent changes, and conventions.
44
5-
SPECTRA is an AI-native test generation and execution framework built in C# / .NET. It has two independent subsystems:
5+
## Quick orientation
66

7-
1. **Spectra.CLI** — AI agent that generates and maintains manual test cases from documentation. Uses the GitHub Copilot SDK as its AI runtime.
8-
2. **Spectra.MCP** — Deterministic MCP execution engine that any LLM orchestrator can drive to execute test suites.
7+
SPECTRA is an AI-native test generation and execution framework written in C# / .NET 8. Three projects under `src/`:
98

10-
Both share **Spectra.Core** for parsing, validation, models, and indexing.
9+
- **`Spectra.CLI`** — CLI tool for test generation, coverage, dashboard, validation. Uses the GitHub Copilot SDK as its sole AI runtime.
10+
- **`Spectra.MCP`** — Deterministic MCP execution server for AI-driven test execution.
11+
- **`Spectra.Core`** — Shared models, parsing, validation, indexing, coverage analyzers.
1112

12-
## Architecture
13+
Tests are split mirror-style across `tests/Spectra.CLI.Tests/`, `tests/Spectra.Core.Tests/`, and `tests/Spectra.MCP.Tests/`.
1314

14-
Read `spec-kit/architecture.md` before making any structural changes. It is the authoritative source for all design decisions.
15+
## Where to look next
1516

16-
Key architectural rules:
17-
- Tests are Markdown files with YAML frontmatter in `tests/{suite}/`
18-
- Documentation lives in `docs/` — the CLI reads from docs, writes to tests
19-
- The MCP server never parses all Markdown files at runtime — it reads `_index.json`
20-
- The AI generation agent reads `docs/_index.md` (document index) instead of scanning all docs at runtime
21-
- The AI agent never writes files directly — all output goes through tool handlers that validate first
22-
- Every MCP response includes `next_expected_action` and is fully self-contained
23-
- State machine transitions are enforced — the MCP server rejects invalid tool call sequences
24-
25-
## Project Structure
26-
27-
```
28-
src/
29-
├── Spectra.CLI/ # .NET CLI application
30-
│ ├── Commands/ # Command handlers (init, validate, index, docs, ai generate, etc.)
31-
│ ├── Agent/ # Copilot SDK integration
32-
│ │ ├── Tools/ # Custom tools for the AI agent
33-
│ │ └── Skills/ # Skill loader
34-
│ ├── Source/ # Document map builder, document index service, source doc reader
35-
│ ├── Index/ # _index.json builder and reader
36-
│ ├── Validation/ # Test case validation, dedup
37-
│ ├── Review/ # Interactive terminal review UI
38-
│ ├── Provider/ # Multi-provider chain with fallback
39-
│ ├── Git/ # Branch/commit operations
40-
│ ├── Config/ # Configuration loader
41-
│ └── IO/ # File writers
42-
├── Spectra.MCP/ # ASP.NET Core MCP server
43-
│ ├── Tools/ # MCP tool handlers
44-
│ ├── Engine/ # State machine, execution queue
45-
│ └── Storage/ # SQLite repository
46-
├── Spectra.Core/ # Shared library
47-
│ ├── Models/ # TestCase, Suite, RunState, etc.
48-
│ ├── Parsing/ # Markdown + YAML frontmatter parser
49-
│ ├── Validation/ # Schema validation rules
50-
│ └── Index/ # Index read/write
51-
└── Spectra.GitHub/ # GitHub integration (Octokit)
52-
53-
spec-kit/ # Architecture specs and ADRs
54-
tests/ # Sample test suites (used for integration testing)
55-
docs/ # Sample docs (used for testing generation)
56-
```
57-
58-
## Conventions
59-
60-
- **Language:** C# 12, .NET 8+
61-
- **Naming:** PascalCase for types and methods, camelCase for locals
62-
- **Async:** All I/O operations are async. Use `Async` suffix on method names.
63-
- **Nullability:** Nullable reference types enabled project-wide
64-
- **Tests:** xUnit for unit tests. Project: `Spectra.Tests`
65-
- **Configuration:** `spectra.config.json` at repo root. Model: `SpectraConfig` in Spectra.Core
66-
67-
## Key Types
68-
69-
When these exist, understand them before modifying related code:
70-
71-
- `TestCase` — Parsed Markdown test with frontmatter metadata
72-
- `TestSuite` — Collection of tests in a folder with an _index.json
73-
- `MetadataIndex` — The _index.json model
74-
- `DocumentMap` — Lightweight listing of all docs in source folder
75-
- `DocumentIndex` — Pre-built document index with rich metadata (sections, entities, tokens, hashes)
76-
- `DocumentIndexEntry` — Per-document metadata in the index
77-
- `DocumentIndexService` — Orchestrates incremental/full index builds
78-
- `ExecutionRun` — A test execution run with state
79-
- `TestHandle` — Opaque reference to a test in a run
80-
- `SpectraConfig` — Root configuration model
81-
82-
## Architecture Reference
83-
84-
**Always read `spec-kit/architecture.md` before making structural changes.** It is the authoritative source for all design decisions: models, MCP tools, state machine, CLI commands, configuration schema, and data flows.
85-
86-
## Spec-Driven Development
87-
88-
This project uses [GitHub Spec Kit](https://github.com/github/spec-kit) for specification-driven development. Specs, plans, and tasks live in `.specify/`. Follow the spec-kit workflow for new features.
89-
90-
## Implementation Guidelines
91-
92-
### Spectra.Core
93-
- Models are plain C# records/classes, no framework dependencies
94-
- Parsing uses Markdig + YamlDotNet
95-
- Validation returns structured results (ParseResult with errors list), never throws
96-
97-
### Spectra.CLI
98-
- Commands use System.CommandLine
99-
- Each command is a thin orchestrator — business logic lives in services
100-
- AI tools are registered via `AIFunctionFactory.Create()` from Microsoft.Extensions.AI
101-
- The Copilot SDK session is created by `CopilotSessionFactory`
102-
- Tool handlers validate before accepting — they are the safety gate
103-
104-
### Spectra.MCP
105-
- MCP tools are ASP.NET Core endpoints using ModelContextProtocol.AspNetCore
106-
- State machine transitions must be validated before executing
107-
- SQLite access goes through a repository pattern
108-
- Every response includes run_status, progress, and next_expected_action
109-
110-
## Active Technologies
111-
- C# 12, .NET 8+ + Spectra.CLI (command integration), Spectra.Core (config, parsing), System.CommandLine (interactive prompts) (004-test-generation-profile)
112-
- File-based (spectra.profile.md at repo root, _profile.md in suites) (004-test-generation-profile)
113-
114-
## Recent Changes
115-
- 010-document-index: Added persistent `docs/_index.md` with per-document metadata, incremental updates via SHA-256 hashing, `spectra docs index` command, auto-refresh before AI generation
116-
- 004-test-generation-profile: Added C# 12, .NET 8+ + Spectra.CLI (command integration), Spectra.Core (config, parsing), System.CommandLine (interactive prompts)
17+
| You want to... | Read |
18+
|---|---|
19+
| Understand the current architecture, file layout, conventions | [`CLAUDE.md`](CLAUDE.md) |
20+
| Build and run locally | [`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md) |
21+
| Use SPECTRA from VS Code Copilot Chat | [`USAGE.md`](USAGE.md) |
22+
| Customize prompts, profiles, branding | [`CUSTOMIZATION.md`](CUSTOMIZATION.md) |
23+
| Project knowledge and component reference | [`PROJECT-KNOWLEDGE.md`](PROJECT-KNOWLEDGE.md) |
24+
| Original architecture design spec | [`architecture-v5.md`](architecture-v5.md) (historical) |
25+
| Contribute | [`CONTRIBUTING.md`](CONTRIBUTING.md) |
26+
| Feature specifications | [`specs/`](specs/) (one directory per feature, numbered 001..NNN) |

CHANGELOG.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,14 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8-
## [Unreleased]
8+
## [1.36.0] - 2026-04-10
99

1010
### Added
11-
- `spectra-quickstart` SKILL (12th bundled SKILL) — workflow-oriented onboarding for Copilot Chat. Triggered by phrases like "help me get started", "tutorial", "walk me through". Presents 12 SPECTRA workflows with example conversations.
12-
- `USAGE.md` — offline workflow reference written to the project root by `spectra init`. Mirrors the quickstart SKILL content in a format suitable for async onboarding, code review, and CI documentation. Hash-tracked by `update-skills` so customizations are preserved.
11+
- **Quickstart SKILL & USAGE.md** (spec 032) — `spectra-quickstart` is the 12th bundled SKILL: workflow-oriented onboarding for Copilot Chat triggered by phrases like "help me get started", "tutorial", "walk me through". Presents 12 SPECTRA workflows with example conversations. Companion `USAGE.md` written to project root by `spectra init` as an offline workflow reference. Both hash-tracked by `update-skills`. Generation and execution agent prompts defer onboarding requests to the new SKILL.
12+
- **Visible default profile format & customization guide** (spec 031) — `profiles/_default.yaml` and `CUSTOMIZATION.md` are now created by `spectra init` and bundled as embedded resources. Profile format is visible/editable instead of hardcoded. New `ProfileFormatLoader.LoadEmbeddedDefaultYaml()` and `LoadEmbeddedCustomizationGuide()` methods.
13+
- **Customizable root prompt templates** (spec 030) — `.spectra/prompts/` directory with 5 markdown templates (behavior-analysis, test-generation, criteria-extraction, critic-verification, test-update) controlling all AI operations. Templates use `{{placeholder}}`, `{{#if}}`, `{{#each}}` syntax. New `analysis.categories` config section with 6 default categories. New `spectra prompts list/show/reset/validate` CLI commands. New `spectra-prompts` SKILL (11th bundled SKILL).
1314
- `ProfileFormatLoader.LoadEmbeddedUsageGuide()` for resolving the bundled `USAGE.md` content.
14-
- Generation and execution agent prompts now defer onboarding requests to the `spectra-quickstart` SKILL.
15+
- Bumped Spectre.Console 0.54.0 → 0.55.0, GitHub.Copilot.SDK 0.2.0 → 0.2.1, Markdig 1.1.1 → 1.1.2 (Dependabot PRs #10, #11, #12).
1516

1617
## [1.35.0] - 2026-04-10
1718

CONTRIBUTING.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,15 +27,15 @@ SPECTRA uses the GitHub Copilot SDK as its sole AI runtime. All AI generation an
2727
- Follow existing code style
2828
- Add tests for new functionality
2929
- Update documentation in `docs/` if the change affects user-facing behavior
30-
- Update `spec-kit/` if the change affects architecture
30+
- Use the spec-kit workflow for new features (`/speckit.specify`, `/speckit.plan`, `/speckit.tasks`, `/speckit.implement`) — specs land under `specs/NNN-feature/`
3131
- Keep PRs focused — one concern per PR
3232

3333
### Documentation
3434

3535
User-facing documentation lives in `docs/`. See the [documentation index](README.md#documentation) for the full list. Each piece of information should live in one place — don't duplicate content between files.
3636

3737
### Architecture Changes
38-
- Propose significant changes as an ADR (Architecture Decision Record) in `spec-kit/adr/`
38+
- Propose significant changes via the spec-kit workflow — start with `/speckit.specify`, then `/speckit.plan`
3939
- Discuss in an Issue before implementing
4040

4141
## Code of Conduct

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ SPECTRA reads your product documentation, generates comprehensive test suites, a
3232

3333
- **Deterministic execution** — An MCP-based execution engine provides a state machine that any AI orchestrator can drive without holding state.
3434

35-
- **Coverage visibility** — Three-dimensional coverage analysis: documentation, requirements, and automation. Visual dashboard included.
35+
- **Coverage visibility** — Three-dimensional coverage analysis: documentation, acceptance criteria, and automation. Visual dashboard included.
3636

3737
- **No migration needed** — Integrates with Azure DevOps, Jira, Teams, Slack through their MCP servers. No data sync. No vendor lock-in.
3838

@@ -58,7 +58,7 @@ Hallucinated steps are caught and rejected automatically.
5858

5959
Three coverage dimensions tracked automatically:
6060
- **Documentation** — which docs have linked tests
61-
- **Requirements** — which requirements are tested
61+
- **Acceptance Criteria** — which criteria are tested
6262
- **Automation** — which tests have automation code
6363

6464
```bash
@@ -86,7 +86,7 @@ Bundled SKILL files let you use SPECTRA through natural language in Copilot Chat
8686
Say "generate test cases for checkout" and the SKILL handles CLI invocation, JSON parsing, and result presentation.
8787

8888
```bash
89-
spectra init # Creates 10 SKILLs + 2 agent prompts
89+
spectra init # Creates 12 SKILLs + 2 agent prompts
9090
spectra update-skills # Update SKILLs when CLI is upgraded
9191
```
9292

@@ -105,7 +105,7 @@ dotnet tool install -g Spectra.CLI
105105
# Initialize (creates config, dirs, SKILL files, agent prompts)
106106
spectra init
107107

108-
# Build the document index (also extracts requirements automatically)
108+
# Build the document index (also extracts acceptance criteria automatically)
109109
spectra docs index
110110

111111
# Generate tests — interactive session with analysis, suggestions, and more

architecture-v5.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
# AI-Native Manual Test Management System — Architecture Specification
22

3+
> **Historical design document.** This is the original architecture spec that guided initial implementation. It is NOT updated as the codebase evolves. For the current state of the system, read [`CLAUDE.md`](CLAUDE.md), [`PROJECT-KNOWLEDGE.md`](PROJECT-KNOWLEDGE.md), and the per-feature specs under [`specs/`](specs/). Treat this file as background reading for deep technical context, not as a source of truth.
4+
35
**Version:** 3.0 (Final Consolidated)
4-
**Status:** Ready for implementation scoping
6+
**Status:** Original design spec — superseded as implementation has evolved
57

68
---
79

0 commit comments

Comments
 (0)