Skip to content

Commit eb2700d

Browse files
Merge branch '025-universal-skill-progress'
Universal progress/result for all SKILL-wrapped commands. ProgressManager shared service, 36 new tests, 1315 total passing.
2 parents dd94224 + b323599 commit eb2700d

30 files changed

+2303
-342
lines changed

CLAUDE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,7 @@ spectra config list-automation-dirs # List dirs with existence s
199199
- **Tests:** xUnit with structured results (never throw on validation errors)
200200

201201
## Recent Changes
202+
- 025-universal-skill-progress: ✅ COMPLETE - Universal progress/result for all SKILL-wrapped commands. New shared `ProgressManager` service (`src/Spectra.CLI/Progress/ProgressManager.cs`) extracts duplicated progress/result file logic from GenerateHandler and DocsIndexHandler. New `ProgressPhases` static class defines phase sequences for 6 command types. All 9 SKILL-wrapped commands now write `.spectra-result.json` on completion. 6 long-running commands (generate, update, docs-index, coverage, extract-criteria, dashboard) write `.spectra-progress.html` with auto-refreshing phase stepper. `ProgressPageWriter` extended with phase configs for update (classifying/updating/verifying), coverage (scanning-tests/analyzing-docs/analyzing-criteria/analyzing-automation), extract-criteria (scanning-docs/extracting/building-index), and dashboard (collecting-data/generating-html). Dynamic title support in progress page header ("SPECTRA — Coverage Analysis", etc.). Summary card renderers for coverage percentages, update counts, criteria counts, dashboard stats. Renamed `AnalyzeCoverageResult.Requirements` → `AcceptanceCriteria` (JSON: `acceptanceCriteria`). New `UpdateResult` model with classification counts. Updated all SKILL files (coverage, criteria, dashboard, validate) with universal 5-step progress flow. Updated generation agent prompt with progress/result instructions for all commands. 36 new tests (19 ProgressManager unit tests, 9 ProgressPageWriter phase/title tests, 5 SKILL flag verification tests, 3 terminology audit tests). 1315 total tests passing.
202203
- 024-docs-skill-coverage-fix: ✅ COMPLETE - Docs index SKILL integration, progress page, coverage fix & terminology rename. Added `.spectra-result.json` and `.spectra-progress.html` to `DocsIndexHandler` (same pattern as generate). New `--skip-criteria` flag skips auto-triggered acceptance criteria extraction. `--no-interaction` passthrough to criteria extraction (runs with defaults). Extended `DocsIndexResult` with new fields (skipped, new, total, criteria). Extended `ProgressPageWriter` for docs-index phases (scanning, indexing, extracting-criteria). New `spectra-docs` SKILL (9th bundled SKILL) with structured tool-call-sequence. Completed "requirements" → "acceptance criteria" rename in all remaining user-facing strings. `CriteriaIndexReader` auto-renames legacy `_requirements.yaml` to `.bak`. Dashboard coverage null-crash fix with zero-state defaults. Updated generation agent prompt for docs index progress-aware flow. Version 1.30.0. 1279 total tests passing.
203204
- 023-criteria-extraction-overhaul: ✅ COMPLETE - Acceptance criteria import & extraction overhaul. Renamed "requirements" to "acceptance criteria" across codebase (CLI, dashboard, reports, SKILLs, agents). Replaced single-prompt AI extraction (truncation-prone) with per-document iterative extraction producing individual `.criteria.yaml` files and `_criteria_index.yaml` master index. SHA-256 incremental extraction skips unchanged docs. Import from YAML/CSV/JSON with auto-column-detection for Jira/ADO exports, AI splitting of compound criteria, RFC 2119 normalization. `--merge` (default) and `--replace` modes. `--list-criteria` with `--source-type`, `--component`, `--priority` filters. Generation auto-loads related criteria as prompt context, produces `criteria` field in test frontmatter. Update flow detects criteria changes (OUTDATED/ORPHANED classification). Dashboard shows per-source-type coverage breakdown. New `spectra-criteria` SKILL (8th). `--extract-requirements` kept as hidden alias. New models: `AcceptanceCriterion`, `CriteriaIndex`, `CriteriaSource`, `AcceptanceCriteriaCoverage`. New services: `CriteriaExtractor`, `CriteriaIndexReader/Writer`, `CriteriaFileReader/Writer`, `CsvCriteriaImporter`, `JsonCriteriaImporter`, `CriteriaMerger`. Live progress HTML page (`.spectra-progress.html`) with auto-refresh for generation status. SKILL/agent names standardized to lowercase-hyphenated format. `browser/openBrowserPage` added to all tool lists. BehaviorAnalyzer retry on failure. MCP `finalize_execution_run` returns instruction to open HTML report. `--no-interaction` on all SKILL CLI commands. `--focus` flag for filtered generation. 38+ new tests, 1280+ total passing.
204205
- 023-copilot-chat-integration: ✅ COMPLETE - Full VS Code Copilot Chat integration for test generation and all CLI features. Added `--suite` option (flag alternative to positional arg) for LLM-friendly syntax. Added `--analyze-only` flag for two-step analyze→approve→generate flow. Automatic batch generation for large counts (batches of 30, writes per batch, index updates per batch). Live progress via `.spectra-result.json` with status lifecycle: analyzing→analyzed→generating→completed/failed. BehaviorAnalyzer timeout increased 30s→2min with proper error handling (was silently failing). 7 bundled SKILLs (generate, coverage, dashboard, validate, list, init-profile, help) with explicit tool-call-sequence format. Both agents (Generation, Execution) handle all SPECTRA commands with help reference. Progress messages from AI tool calls (scanning docs, reading docs, allocating IDs, verifying tests). Per-test verification progress. Critic verification per batch. Smart fallback count accounting for existing tests. Result file with `FileStream.Flush(true)` for Windows NTFS reliability. 1241 total passing.
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Specification Quality Checklist: Universal Progress/Result for SKILL-Wrapped Commands
2+
3+
**Purpose**: Validate specification completeness and quality before proceeding to planning
4+
**Created**: 2026-04-08
5+
**Feature**: [spec.md](../spec.md)
6+
7+
## Content Quality
8+
9+
- [x] No implementation details (languages, frameworks, APIs)
10+
- [x] Focused on user value and business needs
11+
- [x] Written for non-technical stakeholders
12+
- [x] All mandatory sections completed
13+
14+
## Requirement Completeness
15+
16+
- [x] No [NEEDS CLARIFICATION] markers remain
17+
- [x] Requirements are testable and unambiguous
18+
- [x] Success criteria are measurable
19+
- [x] Success criteria are technology-agnostic (no implementation details)
20+
- [x] All acceptance scenarios are defined
21+
- [x] Edge cases are identified
22+
- [x] Scope is clearly bounded
23+
- [x] Dependencies and assumptions identified
24+
25+
## Feature Readiness
26+
27+
- [x] All functional requirements have clear acceptance criteria
28+
- [x] User scenarios cover primary flows
29+
- [x] Feature meets measurable outcomes defined in Success Criteria
30+
- [x] No implementation details leak into specification
31+
32+
## Notes
33+
34+
- All items pass. The spec is derived from a detailed technical design document but has been written at the "what/why" level without leaking implementation specifics.
35+
- The user's original feature description included extensive technical details (C# code, file paths, JSON schemas) which informed the spec's accuracy but were abstracted into business-level requirements.
36+
- Assumptions section documents key design decisions (no concurrent run support, fast vs. long-running command distinction, auto-trigger default for criteria extraction).
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Contract: .spectra-progress.html
2+
3+
**Version**: 1.0
4+
**Consumer**: VS Code Live Preview panel (opened by SKILL via `show preview`)
5+
6+
## Behavior Contract
7+
8+
1. **Created** when a long-running command starts (before first phase)
9+
2. **Updated** at each phase transition and sub-step within phases
10+
3. **Auto-refreshes** every 2 seconds via `<meta http-equiv="refresh" content="2">`
11+
4. **Auto-refresh removed** when command reaches terminal state (completed/failed)
12+
5. **Self-contained** — no external CSS, JS, or fetch calls; all data embedded inline
13+
6. **File links** use `vscode://file/` URI scheme for one-click navigation
14+
15+
## Visual Elements
16+
17+
| Element | Always Present | Description |
18+
|---------|---------------|-------------|
19+
| Title | Yes | "SPECTRA — {Command Name}" in header |
20+
| Phase stepper | Yes | Horizontal steps showing progress through phases |
21+
| Status badge | Yes | Current phase name with spinner (or checkmark/X) |
22+
| Summary cards | When data available | Key metrics in card grid (e.g., "Documents: 15", "Criteria: 45") |
23+
| Error card | On failure only | Red card with error message |
24+
| File links | On completion | Links to generated artifacts |
25+
26+
## Commands That Write Progress HTML
27+
28+
| Command | Title | Phases |
29+
|---------|-------|--------|
30+
| generate | Test Generation | Analyzing → Analyzed → Generating → Completed |
31+
| update | Test Update | Classifying → Updating → Verifying → Completed |
32+
| docs-index | Documentation Index | Scanning → Indexing → Extracting Criteria → Completed |
33+
| coverage | Coverage Analysis | Scanning Tests → Analyzing Docs → Analyzing Criteria → Analyzing Automation → Completed |
34+
| extract-criteria | Criteria Extraction | Scanning Docs → Extracting → Building Index → Completed |
35+
| dashboard | Dashboard Generation | Collecting Data → Generating HTML → Completed |
36+
37+
## Commands That Do NOT Write Progress HTML
38+
39+
validate, list, show, init, import-criteria, list-criteria — these complete too quickly to benefit from a progress page.
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Contract: .spectra-result.json Schema
2+
3+
**Version**: 1.0
4+
**Consumer**: VS Code Copilot Chat SKILLs (via `readFile .spectra-result.json`)
5+
6+
## Base Schema (all commands)
7+
8+
```json
9+
{
10+
"command": "string",
11+
"status": "string",
12+
"timestamp": "string (ISO 8601 UTC)",
13+
"message": "string | null"
14+
}
15+
```
16+
17+
## Per-Command Extensions
18+
19+
### generate
20+
```json
21+
{
22+
"command": "generate",
23+
"status": "analyzing | analyzed | generating | completed | failed",
24+
"suite": "string",
25+
"analysis": { "total_behaviors": 0, "already_covered": 0, "recommended": 0 },
26+
"generation": { "tests_requested": 0, "tests_generated": 0, "tests_written": 0 }
27+
}
28+
```
29+
30+
### update
31+
```json
32+
{
33+
"command": "update",
34+
"status": "classifying | updating | verifying | completed | failed",
35+
"suite": "string",
36+
"testsUpdated": 0,
37+
"testsRemoved": 0,
38+
"testsUnchanged": 0,
39+
"classification": { "upToDate": 0, "outdated": 0, "orphaned": 0, "redundant": 0 }
40+
}
41+
```
42+
43+
### docs-index
44+
```json
45+
{
46+
"command": "docs-index",
47+
"status": "scanning | indexing | extracting-criteria | completed | failed",
48+
"documentsIndexed": 0,
49+
"documentsSkipped": 0,
50+
"documentsTotal": 0,
51+
"criteriaExtracted": 0
52+
}
53+
```
54+
55+
### analyze-coverage
56+
```json
57+
{
58+
"command": "analyze-coverage",
59+
"status": "scanning-tests | analyzing-docs | analyzing-criteria | analyzing-automation | completed | failed",
60+
"documentationCoverage": { "percentage": 0, "covered": 0, "total": 0 },
61+
"acceptanceCriteriaCoverage": { "percentage": 0, "covered": 0, "total": 0 },
62+
"automationCoverage": { "percentage": 0, "covered": 0, "total": 0 }
63+
}
64+
```
65+
66+
### extract-criteria
67+
```json
68+
{
69+
"command": "extract-criteria",
70+
"status": "scanning-docs | extracting | building-index | completed | failed",
71+
"documentsProcessed": 0,
72+
"documentsSkipped": 0,
73+
"criteriaExtracted": 0,
74+
"criteriaNew": 0,
75+
"criteriaUpdated": 0
76+
}
77+
```
78+
79+
### dashboard
80+
```json
81+
{
82+
"command": "dashboard",
83+
"status": "collecting-data | generating-html | completed | failed",
84+
"outputPath": "string",
85+
"suitesIncluded": 0,
86+
"testsIncluded": 0
87+
}
88+
```
89+
90+
### validate
91+
```json
92+
{
93+
"command": "validate",
94+
"status": "completed | failed",
95+
"totalFiles": 0,
96+
"valid": 0,
97+
"errors": []
98+
}
99+
```
100+
101+
## Guarantees
102+
103+
1. File is written atomically (flush to disk) — never partially written
104+
2. File is deleted at command start — stale data never persists
105+
3. File exists after command completion (success or failure)
106+
4. `status` field always present — SKILL can poll for "completed" or "failed"
107+
5. On failure: `"success": false` and `"error": "message"` fields present
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Data Model: Universal Progress/Result
2+
3+
**Date**: 2026-04-08
4+
**Branch**: `025-universal-skill-progress`
5+
6+
## New Entities
7+
8+
### ProgressManager
9+
10+
Shared service encapsulating progress/result file lifecycle.
11+
12+
| Field | Type | Description |
13+
|-------|------|-------------|
14+
| Command | string | Command identifier (e.g., "generate", "docs-index", "coverage") |
15+
| Phases | string[] | Ordered phase names for the phase stepper |
16+
| Title | string | Display title for progress page header |
17+
| ResultPath | string | Path to `.spectra-result.json` (workspace root) |
18+
| ProgressPath | string | Path to `.spectra-progress.html` (workspace root) |
19+
20+
**Lifecycle**: Reset → Start → Update(n) → Complete | Fail
21+
22+
**Methods**:
23+
- `Reset()` — Delete stale files
24+
- `StartAsync(title)` — Create initial progress HTML
25+
- `UpdateAsync(phase, message, summaryData?)` — Update both files
26+
- `CompleteAsync(result)` — Write final result, remove auto-refresh
27+
- `FailAsync(error, partialResult?)` — Write error state
28+
29+
### ProgressPhases (static)
30+
31+
Phase sequences per command type.
32+
33+
| Command | Phases |
34+
|---------|--------|
35+
| Generate | Analyzing → Analyzed → Generating → Completed |
36+
| Update | Classifying → Updating → Verifying → Completed |
37+
| DocsIndex | Scanning → Indexing → Extracting Criteria → Completed |
38+
| Coverage | Scanning Tests → Analyzing Docs → Analyzing Criteria → Analyzing Automation → Completed |
39+
| ExtractCriteria | Scanning Docs → Extracting → Building Index → Completed |
40+
| Dashboard | Collecting Data → Generating HTML → Completed |
41+
42+
## Modified Entities
43+
44+
### AnalyzeCoverageResult (rename field)
45+
46+
| Field | Old Name | New Name | Type |
47+
|-------|----------|----------|------|
48+
| Acceptance criteria section | `Requirements` | `AcceptanceCriteria` | CoverageSection |
49+
50+
JSON output changes: `"requirements": {...}``"acceptanceCriteria": {...}`
51+
52+
## File Artifacts
53+
54+
### .spectra-result.json
55+
56+
Written by all SKILL-wrapped commands. Schema varies by command but always includes base fields:
57+
58+
```json
59+
{
60+
"command": "string",
61+
"status": "string (analyzing|generating|completed|failed|...)",
62+
"timestamp": "ISO 8601 UTC",
63+
"message": "string (optional)"
64+
}
65+
```
66+
67+
### .spectra-progress.html
68+
69+
Written by long-running commands only. Self-contained HTML with:
70+
- Auto-refresh meta tag (2s interval, removed on completion/failure)
71+
- Phase stepper showing progress through command phases
72+
- Summary cards with command-specific data
73+
- Status badge with current operation message
74+
- Error card (on failure)
75+
- VS Code file links for generated artifacts
76+
77+
## State Transitions
78+
79+
```
80+
[Reset] → files deleted
81+
82+
[Start] → progress HTML created (phase 1 active, auto-refresh ON)
83+
84+
[Update]* → progress HTML updated (phase N active), result JSON updated
85+
86+
[Complete] → final result JSON, auto-refresh OFF, last phase marked done
87+
OR
88+
[Fail] → error result JSON, auto-refresh OFF, current phase marked failed
89+
```
90+
91+
## Relationships
92+
93+
- `ProgressManager` → uses `ProgressPageWriter` (existing static class)
94+
- `ProgressManager` → uses `JsonResultWriter` serialization options
95+
- Each command handler → owns one `ProgressManager` instance
96+
- `ProgressPageWriter` → reads `ProgressPhases` to render correct stepper

0 commit comments

Comments
 (0)