Skip to content

Commit 201b5f1

Browse files
angelovstantonangelovstanton
committed
feat: add spectra-update SKILL (10th bundled SKILL) & version 1.35.0
Add the spectra-update SKILL for test update workflow via Copilot Chat, completing the Chat-first lifecycle for test maintenance. Both agents now delegate update requests to the SKILL instead of handling inline. - New embedded SKILL resource: spectra-update.md (5-step progress flow) - Extended UpdateResult with success, totalTests, testsFlagged, flaggedTests, duration fields + FlaggedTestEntry class - Generation agent: replaced inline update section with delegation row - Execution agent: added update delegation row - 6 new tests (SKILL content, format, agents delegation) - Documentation: SKILL count 9→10 across all files - 1360 total tests passing
1 parent 0ad1380 commit 201b5f1

File tree

18 files changed

+915
-26
lines changed

18 files changed

+915
-26
lines changed

‎CHANGELOG.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [1.35.0] - 2026-04-10
9+
10+
### Added
11+
- `spectra-update` SKILL (10th bundled SKILL) for test update workflow via Copilot Chat
12+
- Agent delegation tables updated for update command routing
13+
- `UpdateResult` extended with `totalTests`, `testsFlagged`, `flaggedTests`, `duration`, `success` fields
14+
15+
## [1.34.6] - 2026-04-09
16+
17+
### Fixed
18+
- Dashboard test file path resolution & version bump
19+
20+
## [1.34.0] - 2026-04-08
21+
22+
### Fixed
23+
- Criteria coverage, index generation, dashboard SKILL & CI pragma fixes
24+
25+
## [1.33.0] - 2026-04-07
26+
27+
### Fixed
28+
- Criteria generation, polling, progress caching & CI build fixes
29+
30+
## [1.32.0] - 2026-04-06
31+
32+
### Changed
33+
- Refined SPECTRA agents/skills CLI workflows
34+
35+
## [1.31.0] - 2026-04-05
36+
37+
### Added
38+
- Coverage semantics fix & criteria-generation pipeline (spec 028)
39+
- `TestCaseParser` now propagates `Criteria` field from frontmatter to `TestCase`
40+
- Criteria loading wired into `GenerateHandler` for per-doc `.criteria.yaml` context
41+
42+
## [1.30.0] - 2026-04-04
43+
44+
### Added
45+
- Docs index SKILL integration, progress page, coverage fix & terminology rename (spec 024)
46+
- `spectra-docs` SKILL (9th bundled SKILL) with structured tool-call-sequence
47+
- `--skip-criteria` flag for docs index command
48+
49+
### Fixed
50+
- Dashboard coverage null-crash fix with zero-state defaults

‎CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,7 @@ spectra config list-automation-dirs # List dirs with existence s
199199
- **Tests:** xUnit with structured results (never throw on validation errors)
200200

201201
## Recent Changes
202+
- 029-spectra-update-skill: ✅ COMPLETE - Added spectra-update SKILL (10th bundled SKILL) for test update workflow via Copilot Chat. SKILL wraps `spectra ai update` with progress page, result file, classification breakdown (UP_TO_DATE, OUTDATED, ORPHANED, REDUNDANT). Agent delegation tables updated (both generation and execution agents delegate update requests to SKILL). Extended `UpdateResult` with `success`, `totalTests`, `testsFlagged`, `flaggedTests`, `duration` fields. Generation agent inline update section replaced with delegation row. 6 new tests (SKILL content, step format, do-NOTHING instruction, tools list, agent delegation). Documentation updated (SKILL count 9→10). Version 1.35.0.
202203
- 028-coverage-criteria-fix: ✅ COMPLETE - Coverage semantics fix & criteria-generation pipeline. Fixed `TestCaseParser` to propagate `Criteria` field from frontmatter to `TestCase` (was missing). Wired criteria loading into `GenerateHandler`: loads per-doc `.criteria.yaml` files matching suite name and component, formats as context string, passes to `CopilotGenerationAgent.BuildFullPrompt()` via new `criteriaContext` parameter on `GenerateTestsAsync()`. `TestFileWriter` now always writes `criteria: []` field (even when empty) for visibility. Audit confirmed coverage analyzers already correct: `DocumentationCoverageAnalyzer` checks test existence via `source_refs`, `AcceptanceCriteriaCoverageAnalyzer` reads both `criteria` and legacy `requirements` fields. 4 new regression tests. 1354 total tests passing.
203204
- 027-skill-agent-dedup: ✅ COMPLETE - SKILL/Agent prompt deduplication. Refactored both agents to delegate CLI tasks to authoritative SKILL files instead of duplicating instructions. Execution agent reduced from ~400 to 120 lines, generation agent from ~219 to 81 lines. Agents now contain delegation tables pointing to `spectra-*` SKILLs for dashboard, coverage, criteria, validate, list, and docs index tasks. Fixed SKILL inconsistencies: replaced "Tool call N" with "Step N" format in spectra-list, spectra-init-profile, spectra-validate; added `--no-interaction --output-format json --verbosity quiet` flags to spectra-list and spectra-init-profile; added "do NOTHING between runInTerminal and awaitTerminal" instruction to spectra-coverage and spectra-dashboard; added incremental vs force note to spectra-docs. Extended spectra-help SKILL with acceptance criteria and documentation index sections. Removed redundant "NEVER" warnings from execution agent (kept askQuestion and fabrication warnings). 6 new tests (agent line count limits, no CLI block duplication, step format consistency, no terminalLastCommand, help completeness). 1350 total tests passing.
204205
- 026-criteria-folder-coverage-fix: ✅ COMPLETE - Criteria folder rename, index exclusion & coverage fix. Renamed default criteria directory from `docs/requirements/` to `docs/criteria/` across all config defaults, init handler, update handler fallbacks, and documentation. Auto-migration logic in `AnalyzeHandler.MigrateCriteriaFolderAsync()` renames `docs/requirements/` → `docs/criteria/` on first run, updates `spectra.config.json` paths, and deletes stale `_index.criteria.yaml`. Added `ShouldSkipDocument()` filter to exclude `_index.md`, `_index.yaml`, `_index.json`, `.criteria.yaml`, and `_criteria_index.yaml` files from criteria extraction (prevents duplicate criteria from metadata files). Fixed dashboard acceptance criteria coverage: `DataCollector` now uses `AcceptanceCriteriaCoverageAnalyzer` with per-document `.criteria.yaml` file enumeration instead of legacy `AcceptanceCriteriaParser`, and reads both `Requirements` and `Criteria` fields from test index entries. Fixed `app.js` unit label from `acceptance_criteria` to `criteria` for proper grammar. Updated empty-state message to guide users to `spectra ai analyze --extract-criteria`. Updated all documentation (CLAUDE.md, PROJECT-KNOWLEDGE.md, cli-reference.md, coverage.md, getting-started.md, configuration.md). Version 1.32.0. 29 new tests (10 document skip tests, 7 coverage analyzer tests, 5 migration tests, 3 config default tests, 4 terminology audit updates). 1344 total tests passing.

‎PROJECT-KNOWLEDGE.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ User → VS Code Copilot Chat → MCP Server → Test Execution → Reports
2525
| Component | Purpose |
2626
|-----------|---------|
2727
| **dashboard-site/** | Static HTML/JS dashboard template with D3.js visualizations |
28-
| **SKILLs** (9 files) | VS Code Copilot Chat integration — each SKILL wraps CLI commands |
28+
| **SKILLs** (10 files) | VS Code Copilot Chat integration — each SKILL wraps CLI commands |
2929
| **Agents** (2 files) | Copilot Chat agent prompts for generation and execution workflows |
3030

3131
## Technology Stack
@@ -187,10 +187,11 @@ criteria:
187187

188188
## VS Code Copilot Chat Integration
189189

190-
### 9 Bundled SKILLs
190+
### 10 Bundled SKILLs
191191
| SKILL | Purpose |
192192
|-------|---------|
193193
| `spectra-generate` | Test generation (analyze → approve → generate flow) |
194+
| `spectra-update` | Test updates (classify → review → apply changes) |
194195
| `spectra-coverage` | Coverage analysis |
195196
| `spectra-dashboard` | Dashboard generation and opening |
196197
| `spectra-validate` | Test validation |
@@ -203,8 +204,8 @@ criteria:
203204
### 2 Agent Prompts (Delegation Model)
204205
| Agent | Purpose |
205206
|-------|---------|
206-
| `spectra-generation` (~81 lines) | Primary: test generation and update. Delegates all other CLI tasks to corresponding SKILLs via delegation table. |
207-
| `spectra-execution` (~120 lines) | Test execution via MCP tools. Delegates all CLI tasks (dashboard, coverage, criteria, validate, list, docs index) to corresponding SKILLs via delegation table. |
207+
| `spectra-generation` (~81 lines) | Primary: test generation and update. Delegates all other CLI tasks (update, dashboard, coverage, criteria, validate, list, docs index) to corresponding SKILLs via delegation table. |
208+
| `spectra-execution` (~120 lines) | Test execution via MCP tools. Delegates all CLI tasks (update, dashboard, coverage, criteria, validate, list, docs index) to corresponding SKILLs via delegation table. |
208209

209210
Agents are **routers**, not executors. Each CLI command has exactly one source of truth — its SKILL file. Agents reference SKILLs by name (e.g., "Follow the `spectra-dashboard` SKILL") instead of duplicating CLI instructions.
210211

@@ -301,6 +302,7 @@ Three-section unified coverage with distinct semantics:
301302

302303
| # | Feature | Key Changes |
303304
|---|---------|-------------|
305+
| 029 | spectra-update SKILL (10th) | Agent delegation, documentation sync, version 1.35.0 |
304306
| 028 | Coverage & Criteria Pipeline | Fixed criteria propagation in parser, wired criteria into generation pipeline, always write criteria: [] |
305307
| 027 | SKILL/Agent Deduplication | Agents delegate to SKILLs, execution ~120 lines, generation ~81 lines, SKILL consistency fixes |
306308
| 024 | Docs Index SKILL & Coverage Fix | 9th SKILL (spectra-docs), result/progress files, --skip-criteria, terminology fix |

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ Bundled SKILL files let you use SPECTRA through natural language in Copilot Chat
8686
Say "generate test cases for checkout" and the SKILL handles CLI invocation, JSON parsing, and result presentation.
8787

8888
```bash
89-
spectra init # Creates 6 SKILLs + 2 agent prompts
89+
spectra init # Creates 10 SKILLs + 2 agent prompts
9090
spectra update-skills # Update SKILLs when CLI is upgraded
9191
```
9292

‎specs/029-update-skill-docs-sync/checklists/requirements.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Specification Quality Checklist: SPECTRA Update SKILL + Documentation Sync
2+
3+
**Purpose**: Validate specification completeness and quality before proceeding to planning
4+
**Created**: 2026-04-10
5+
**Feature**: [spec.md](../spec.md)
6+
7+
## Content Quality
8+
9+
- [x] No implementation details (languages, frameworks, APIs)
10+
- [x] Focused on user value and business needs
11+
- [x] Written for non-technical stakeholders
12+
- [x] All mandatory sections completed
13+
14+
## Requirement Completeness
15+
16+
- [x] No [NEEDS CLARIFICATION] markers remain
17+
- [x] Requirements are testable and unambiguous
18+
- [x] Success criteria are measurable
19+
- [x] Success criteria are technology-agnostic (no implementation details)
20+
- [x] All acceptance scenarios are defined
21+
- [x] Edge cases are identified
22+
- [x] Scope is clearly bounded
23+
- [x] Dependencies and assumptions identified
24+
25+
## Feature Readiness
26+
27+
- [x] All functional requirements have clear acceptance criteria
28+
- [x] User scenarios cover primary flows
29+
- [x] Feature meets measurable outcomes defined in Success Criteria
30+
- [x] No implementation details leak into specification
31+
32+
## Notes
33+
34+
- All items pass. Spec is ready for `/speckit.plan`.
35+
- FR-015 and FR-016 are verification requirements (confirming existing infrastructure from spec 025) rather than new implementation.
36+
- The spec references specific file names (SkillContent.cs, SkillsManifest.cs, etc.) as key entities, which is appropriate since this is a well-established codebase with known conventions.

‎specs/029-update-skill-docs-sync/data-model.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Data Model: SPECTRA Update SKILL + Documentation Sync
2+
3+
**Date**: 2026-04-10 | **Branch**: `029-update-skill-docs-sync`
4+
5+
## Entities
6+
7+
### UpdateResult (extended)
8+
9+
Existing model at `src/Spectra.CLI/Results/UpdateResult.cs`. Extends `CommandResult`.
10+
11+
**Existing fields** (no change):
12+
| Field | Type | JSON Key | Description |
13+
|-------|------|----------|-------------|
14+
| Command | string | command | "update" |
15+
| Status | string | status | "completed" / "failed" |
16+
| Timestamp | string | timestamp | ISO 8601 |
17+
| Message | string? | message | Optional status message |
18+
| Suite | string? | suite | Target suite name |
19+
| TestsUpdated | int | testsUpdated | Count of rewritten tests |
20+
| TestsRemoved | int | testsRemoved | Count of removed tests |
21+
| TestsUnchanged | int | testsUnchanged | Count of unchanged tests |
22+
| Classification | UpdateClassificationCounts? | classification | Breakdown by classification type |
23+
| FilesModified | IReadOnlyList<string>? | filesModified | List of modified file paths |
24+
| FilesDeleted | IReadOnlyList<string>? | filesDeleted | List of deleted file paths |
25+
26+
**New fields** (to add):
27+
| Field | Type | JSON Key | Description |
28+
|-------|------|----------|-------------|
29+
| Success | bool | success | Whether command completed without errors |
30+
| TotalTests | int | totalTests | Total tests analyzed (sum of classifications) |
31+
| TestsFlagged | int | testsFlagged | Count of tests flagged for review (orphaned + redundant) |
32+
| FlaggedTests | IReadOnlyList<FlaggedTestEntry>? | flaggedTests | Details of flagged tests |
33+
| Duration | string? | duration | Formatted duration "HH:mm:ss" |
34+
35+
### FlaggedTestEntry (new)
36+
37+
New model class for detailed flagged test information.
38+
39+
| Field | Type | JSON Key | Description |
40+
|-------|------|----------|-------------|
41+
| Id | string | id | Test case ID (e.g., "TC-107") |
42+
| Title | string | title | Test case title |
43+
| Classification | string | classification | "ORPHANED" or "REDUNDANT" |
44+
| Reason | string | reason | Human-readable explanation |
45+
46+
### UpdateClassificationCounts (no change)
47+
48+
Already complete with `UpToDate`, `Outdated`, `Orphaned`, `Redundant` fields.
49+
50+
## SKILL Content Entity
51+
52+
The SKILL file is a Markdown document with YAML frontmatter, stored as an embedded resource.
53+
54+
**Frontmatter schema** (standard across all SKILLs):
55+
| Field | Value |
56+
|-------|-------|
57+
| name | spectra-update |
58+
| description | Update existing test cases after documentation or acceptance criteria changes. |
59+
| tools | [{{READONLY_TOOLS}}] |
60+
| model | GPT-4o |
61+
| disable-model-invocation | true |
62+
63+
## State Transitions
64+
65+
No new state transitions. The SKILL delegates to the existing `UpdateHandler` which manages its own state machine via `ProgressManager`:
66+
67+
```
68+
classifying → updating → verifying → completed
69+
```
70+
71+
These phases are already defined in `ProgressPhases.Update`.

‎specs/029-update-skill-docs-sync/plan.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Implementation Plan: SPECTRA Update SKILL + Documentation Sync
2+
3+
**Branch**: `029-update-skill-docs-sync` | **Date**: 2026-04-10 | **Spec**: [spec.md](spec.md)
4+
**Input**: Feature specification from `/specs/029-update-skill-docs-sync/spec.md`
5+
6+
## Summary
7+
8+
Add the 10th bundled SKILL (`spectra-update`) wrapping `spectra ai update` for Copilot Chat integration. Update agent delegation tables so both agents route update requests to the SKILL. Extend `UpdateResult` with missing fields. Sync all documentation to reflect 10 SKILLs.
9+
10+
## Technical Context
11+
12+
**Language/Version**: C# 12, .NET 8+
13+
**Primary Dependencies**: System.Text.Json (serialization), Spectre.Console (terminal UX)
14+
**Storage**: File-based (embedded resources for SKILL content, `.spectra-result.json` / `.spectra-progress.html` for progress)
15+
**Testing**: xUnit (1354+ existing tests across 3 test projects)
16+
**Target Platform**: Windows, macOS, Linux (cross-platform .NET CLI tool)
17+
**Project Type**: CLI tool with VS Code Copilot Chat integration via bundled SKILLs
18+
**Performance Goals**: N/A (no runtime performance changes)
19+
**Constraints**: SKILL file must follow exact conventions (5-step flow, flags, step format, wait instruction) for Chat agent compatibility
20+
**Scale/Scope**: ~150 lines new code, ~100 lines test code, ~50 lines documentation changes
21+
22+
## Constitution Check
23+
24+
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
25+
26+
| Principle | Status | Notes |
27+
|-----------|--------|-------|
28+
| I. GitHub as Source of Truth | PASS | SKILL files are stored in Git at `.github/skills/`, embedded in the CLI binary as resources |
29+
| II. Deterministic Execution | PASS | No execution engine changes; SKILL delegates to existing deterministic CLI command |
30+
| III. Orchestrator-Agnostic Design | PASS | SKILL uses standard tools format compatible with any VS Code Chat participant |
31+
| IV. CLI-First Interface | PASS | SKILL wraps existing CLI command `spectra ai update`; no new functionality added outside CLI |
32+
| V. Simplicity (YAGNI) | PASS | No new abstractions; follows exact pattern of 9 existing SKILLs. Embedded resource auto-discovery means minimal code changes |
33+
34+
All gates pass. No violations to justify.
35+
36+
## Project Structure
37+
38+
### Documentation (this feature)
39+
40+
```text
41+
specs/029-update-skill-docs-sync/
42+
├── plan.md # This file
43+
├── research.md # Phase 0 output
44+
├── data-model.md # Phase 1 output
45+
├── quickstart.md # Phase 1 output
46+
└── tasks.md # Phase 2 output (/speckit.tasks)
47+
```
48+
49+
### Source Code (repository root)
50+
51+
```text
52+
src/Spectra.CLI/Skills/Content/Skills/
53+
├── spectra-update.md # NEW: 10th embedded SKILL resource
54+
55+
src/Spectra.CLI/Skills/
56+
├── SkillContent.cs # MODIFY: Add Update property
57+
├── SkillsManifest.cs # NO CHANGE: Auto-discovers via SkillContent.All
58+
59+
src/Spectra.CLI/Skills/Content/Agents/
60+
├── spectra-generation.agent.md # MODIFY: Move update section to delegation table
61+
├── spectra-execution.agent.md # MODIFY: Add update delegation row
62+
63+
src/Spectra.CLI/Skills/
64+
├── AgentContent.cs # NO CHANGE: Auto-loads from embedded resources
65+
66+
src/Spectra.CLI/Commands/Init/
67+
├── InitHandler.cs # NO CHANGE: Loops through SkillContent.All (auto-discovers)
68+
69+
src/Spectra.CLI/Results/
70+
├── UpdateResult.cs # MODIFY: Add missing fields (totalTests, testsFlagged, flaggedTests, duration, success)
71+
72+
tests/Spectra.CLI.Tests/Skills/
73+
├── SkillsManifestTests.cs # MODIFY: Update test count assertions, add spectra-update tests
74+
```
75+
76+
**Structure Decision**: Follows existing project structure exactly. The SKILL is an embedded .md resource auto-discovered by `SkillResourceLoader`. No new directories or projects needed.
77+
78+
## Complexity Tracking
79+
80+
No violations. All changes follow established patterns.

‎specs/029-update-skill-docs-sync/quickstart.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Quickstart: SPECTRA Update SKILL + Documentation Sync
2+
3+
**Date**: 2026-04-10 | **Branch**: `029-update-skill-docs-sync`
4+
5+
## Implementation Steps
6+
7+
### Step 1: Create the SKILL embedded resource
8+
9+
Create `src/Spectra.CLI/Skills/Content/Skills/spectra-update.md` following the pattern of `spectra-docs.md` (long-running SKILL with 5-step flow). Use `tools: [{{READONLY_TOOLS}}]` frontmatter.
10+
11+
### Step 2: Add property to SkillContent.cs
12+
13+
Add `public static string Update => All["spectra-update"];` to `SkillContent.cs`. The resource loader auto-discovers the `.md` file.
14+
15+
### Step 3: Update agent delegation files
16+
17+
- `spectra-generation.agent.md`: Remove the inline "Update tests" section (lines 56-67). Add a delegation row: `| Update tests | spectra-update | spectra ai update --suite {suite} --no-interaction --output-format json --verbosity quiet |`
18+
- `spectra-execution.agent.md`: Add delegation row: `| Update tests | spectra-update | spectra ai update --suite {suite} --no-interaction --output-format json --verbosity quiet |`
19+
20+
### Step 4: Extend UpdateResult model
21+
22+
Add `Success`, `TotalTests`, `TestsFlagged`, `FlaggedTests`, `Duration` fields. Add `FlaggedTestEntry` class.
23+
24+
### Step 5: Update UpdateHandler to populate new fields
25+
26+
Set the new fields when constructing the `UpdateResult` at completion.
27+
28+
### Step 6: Update tests
29+
30+
- Update SKILL count assertions from 9 to 10
31+
- Add spectra-update content assertions
32+
- Add agent delegation assertions
33+
34+
### Step 7: Update documentation
35+
36+
- PROJECT-KNOWLEDGE.md: SKILL count + table + completed specs
37+
- CLAUDE.md: Recent Changes entry
38+
- README.md: SKILL count
39+
- CHANGELOG.md: New entry
40+
41+
## Verification
42+
43+
```bash
44+
# Build
45+
dotnet build
46+
47+
# Run all tests
48+
dotnet test
49+
50+
# Verify SKILL count
51+
# Tests should assert 10 SKILLs in SkillContent.All
52+
```
53+
54+
## Key Files
55+
56+
| File | Action |
57+
|------|--------|
58+
| `src/Spectra.CLI/Skills/Content/Skills/spectra-update.md` | CREATE |
59+
| `src/Spectra.CLI/Skills/SkillContent.cs` | MODIFY (add property) |
60+
| `src/Spectra.CLI/Skills/Content/Agents/spectra-generation.agent.md` | MODIFY (delegation) |
61+
| `src/Spectra.CLI/Skills/Content/Agents/spectra-execution.agent.md` | MODIFY (delegation) |
62+
| `src/Spectra.CLI/Results/UpdateResult.cs` | MODIFY (add fields) |
63+
| `src/Spectra.CLI/Commands/Update/UpdateHandler.cs` | MODIFY (populate fields) |
64+
| `tests/Spectra.CLI.Tests/Skills/SkillsManifestTests.cs` | MODIFY (update + new tests) |
65+
| `PROJECT-KNOWLEDGE.md` | MODIFY (SKILL count, table) |
66+
| `CLAUDE.md` | MODIFY (recent changes) |
67+
| `README.md` | MODIFY (SKILL count) |
68+
| `CHANGELOG.md` | MODIFY (new entry) |

0 commit comments

Comments
 (0)
⚡