Skip to content

Commit dd94224

Browse files
angelovstantonangelovstanton
committed
docs: update all docs for spec 024 changes
- Update test counts (462/466/351 = 1279 total) - Update SKILL count 8→9 everywhere (CLAUDE.md, skills-integration.md, getting-started.md, PROJECT-KNOWLEDGE.md) - Add spectra-docs SKILL to all SKILL tables - Document --skip-criteria flag and SKILL/CI mode for docs index - Complete requirements→acceptance criteria terminology in coverage.md, document-index.md - Add 024 entry to Recent Changes and feature spec tables - Update config references _requirements.yaml→_criteria_index.yaml
1 parent d9654bf commit dd94224

File tree

8 files changed

+74
-52
lines changed

8 files changed

+74
-52
lines changed

‎CLAUDE.md‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -82,16 +82,16 @@ dashboard-site/ # Static dashboard template
8282
└── access-denied.html # Auth error page
8383
8484
tests/
85-
├── Spectra.Core.Tests/ # Unit tests (349 tests)
85+
├── Spectra.Core.Tests/ # Unit tests (462 tests)
8686
│ ├── Coverage/ # AutomationScanner, LinkReconciler, Calculator, DocCoverageAnalyzer, ReqCoverageAnalyzer, AutoLinkService tests
8787
│ ├── Index/ # DocumentIndexReader, DocumentIndexWriter tests
8888
│ └── Parsing/ # DocumentIndexExtractor, RequirementsParser, FrontmatterUpdater tests
89-
├── Spectra.CLI.Tests/ # Integration tests (329 tests)
89+
├── Spectra.CLI.Tests/ # Integration tests (466 tests)
9090
│ ├── Commands/ # DocsIndexCommand tests
9191
│ ├── Dashboard/ # DataCollector, Generator tests
9292
│ ├── Source/ # DocumentIndexService tests
9393
│ └── Coverage/ # CoverageReportWriter (unified), CoverageAnalysis tests
94-
├── Spectra.MCP.Tests/ # MCP server tests (306 tests)
94+
├── Spectra.MCP.Tests/ # MCP server tests (351 tests)
9595
│ ├── Tools/ # Individual tool tests
9696
│ ├── Integration/ # Full execution flow tests
9797
│ └── Reports/ # Report generation tests
@@ -146,9 +146,11 @@ spectra dashboard --dry-run # Preview without generating
146146
spectra dashboard --preview # Sample data + branding verification
147147
spectra dashboard --output ./site --output-format json # JSON output for SKILL
148148

149-
# Documentation Index (010-document-index)
150-
spectra docs index # Incremental update + auto-extract requirements
151-
spectra docs index --force # Full rebuild + auto-extract requirements
149+
# Documentation Index (010-document-index + 024-docs-skill-coverage-fix)
150+
spectra docs index # Incremental update + auto-extract acceptance criteria
151+
spectra docs index --force # Full rebuild + auto-extract acceptance criteria
152+
spectra docs index --skip-criteria # Index only, skip criteria extraction
153+
spectra docs index --no-interaction --output-format json # SKILL/CI mode (writes .spectra-result.json + .spectra-progress.html)
152154

153155
# Coverage Analysis (003 + unified coverage overhaul)
154156
spectra ai analyze --coverage # Unified three-section report (doc, req, auto)
@@ -197,6 +199,7 @@ spectra config list-automation-dirs # List dirs with existence s
197199
- **Tests:** xUnit with structured results (never throw on validation errors)
198200

199201
## Recent Changes
202+
- 024-docs-skill-coverage-fix: ✅ COMPLETE - Docs index SKILL integration, progress page, coverage fix & terminology rename. Added `.spectra-result.json` and `.spectra-progress.html` to `DocsIndexHandler` (same pattern as generate). New `--skip-criteria` flag skips auto-triggered acceptance criteria extraction. `--no-interaction` passthrough to criteria extraction (runs with defaults). Extended `DocsIndexResult` with new fields (skipped, new, total, criteria). Extended `ProgressPageWriter` for docs-index phases (scanning, indexing, extracting-criteria). New `spectra-docs` SKILL (9th bundled SKILL) with structured tool-call-sequence. Completed "requirements" → "acceptance criteria" rename in all remaining user-facing strings. `CriteriaIndexReader` auto-renames legacy `_requirements.yaml` to `.bak`. Dashboard coverage null-crash fix with zero-state defaults. Updated generation agent prompt for docs index progress-aware flow. Version 1.30.0. 1279 total tests passing.
200203
- 023-criteria-extraction-overhaul: ✅ COMPLETE - Acceptance criteria import & extraction overhaul. Renamed "requirements" to "acceptance criteria" across codebase (CLI, dashboard, reports, SKILLs, agents). Replaced single-prompt AI extraction (truncation-prone) with per-document iterative extraction producing individual `.criteria.yaml` files and `_criteria_index.yaml` master index. SHA-256 incremental extraction skips unchanged docs. Import from YAML/CSV/JSON with auto-column-detection for Jira/ADO exports, AI splitting of compound criteria, RFC 2119 normalization. `--merge` (default) and `--replace` modes. `--list-criteria` with `--source-type`, `--component`, `--priority` filters. Generation auto-loads related criteria as prompt context, produces `criteria` field in test frontmatter. Update flow detects criteria changes (OUTDATED/ORPHANED classification). Dashboard shows per-source-type coverage breakdown. New `spectra-criteria` SKILL (8th). `--extract-requirements` kept as hidden alias. New models: `AcceptanceCriterion`, `CriteriaIndex`, `CriteriaSource`, `AcceptanceCriteriaCoverage`. New services: `CriteriaExtractor`, `CriteriaIndexReader/Writer`, `CriteriaFileReader/Writer`, `CsvCriteriaImporter`, `JsonCriteriaImporter`, `CriteriaMerger`. Live progress HTML page (`.spectra-progress.html`) with auto-refresh for generation status. SKILL/agent names standardized to lowercase-hyphenated format. `browser/openBrowserPage` added to all tool lists. BehaviorAnalyzer retry on failure. MCP `finalize_execution_run` returns instruction to open HTML report. `--no-interaction` on all SKILL CLI commands. `--focus` flag for filtered generation. 38+ new tests, 1280+ total passing.
201204
- 023-copilot-chat-integration: ✅ COMPLETE - Full VS Code Copilot Chat integration for test generation and all CLI features. Added `--suite` option (flag alternative to positional arg) for LLM-friendly syntax. Added `--analyze-only` flag for two-step analyze→approve→generate flow. Automatic batch generation for large counts (batches of 30, writes per batch, index updates per batch). Live progress via `.spectra-result.json` with status lifecycle: analyzing→analyzed→generating→completed/failed. BehaviorAnalyzer timeout increased 30s→2min with proper error handling (was silently failing). 7 bundled SKILLs (generate, coverage, dashboard, validate, list, init-profile, help) with explicit tool-call-sequence format. Both agents (Generation, Execution) handle all SPECTRA commands with help reference. Progress messages from AI tool calls (scanning docs, reading docs, allocating IDs, verifying tests). Per-test verification progress. Critic verification per batch. Smart fallback count accounting for existing tests. Result file with `FileStream.Flush(true)` for Windows NTFS reliability. 1241 total passing.
202205
- 022-bundled-skills: ✅ COMPLETE - Bundled SKILL files and agent prompts for Copilot Chat integration. 6 SKILL files (generate, coverage, dashboard, validate, list, init-profile) and 2 agent prompts (execution, generation) created by `spectra init` in `.github/skills/` and `.github/agents/`. Each SKILL contains CLI commands with `--output-format json --verbosity quiet`, JSON parsing instructions, and example user requests. New `SkillContent`, `AgentContent` static classes with bundled content. New `SkillsManifest` with SHA-256 hash tracking for safe updates. New `spectra update-skills` command (updates unmodified files, skips user-customized). New `--skip-skills` flag on init. New `FileHasher` utility. 10 new tests, 1241 total passing.

‎PROJECT-KNOWLEDGE.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ User → VS Code Copilot Chat → MCP Server → Test Execution → Reports
2525
| Component | Purpose |
2626
|-----------|---------|
2727
| **dashboard-site/** | Static HTML/JS dashboard template with D3.js visualizations |
28-
| **SKILLs** (8 files) | VS Code Copilot Chat integration — each SKILL wraps CLI commands |
28+
| **SKILLs** (9 files) | VS Code Copilot Chat integration — each SKILL wraps CLI commands |
2929
| **Agents** (2 files) | Copilot Chat agent prompts for generation and execution workflows |
3030

3131
## Technology Stack
@@ -187,7 +187,7 @@ criteria:
187187

188188
## VS Code Copilot Chat Integration
189189

190-
### 8 Bundled SKILLs
190+
### 9 Bundled SKILLs
191191
| SKILL | Purpose |
192192
|-------|---------|
193193
| `spectra-generate` | Test generation (analyze → approve → generate flow) |
@@ -198,6 +198,7 @@ criteria:
198198
| `spectra-init-profile` | Generation profile setup |
199199
| `spectra-help` | Command reference |
200200
| `spectra-criteria` | Criteria extraction, import, listing |
201+
| `spectra-docs` | Documentation indexing with progress page |
201202

202203
### 2 Agent Prompts
203204
| Agent | Purpose |
@@ -209,7 +210,7 @@ criteria:
209210
- **Name**: lowercase-hyphenated (e.g., `spectra-generate`)
210211
- **Tools**: Include `browser/openBrowserPage` for preview capability
211212
- **CLI flags**: Always include `--no-interaction` to prevent terminal prompts
212-
- **Progress**: Generation commands use `.spectra-progress.html` for live status
213+
- **Progress**: Generation and docs index commands use `.spectra-progress.html` for live status
213214
- **Steps**: Use `**Step N**` format (not `### Tool call N:`)
214215
- **Preview**: Use `show preview {file}` to open files in VS Code (not `start`)
215216

@@ -296,6 +297,7 @@ Three-section unified coverage:
296297

297298
| # | Feature | Key Changes |
298299
|---|---------|-------------|
300+
| 024 | Docs Index SKILL & Coverage Fix | 9th SKILL (spectra-docs), result/progress files, --skip-criteria, terminology fix |
299301
| 023 | Criteria Extraction Overhaul | Per-doc extraction, import, rename requirements→criteria, progress page |
300302
| 023 | Copilot Chat Integration | SKILLs, agents, result file polling, batch generation |
301303
| 022 | Bundled Skills | 8 SKILLs + 2 agents embedded as resources, hash-tracked updates |

‎docs/PROJECT-KNOWLEDGE.md‎

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ spectra init --skip-skills # Skip SKILL/agent file creation
129129
spectra init --no-interactive # Non-interactive mode
130130
```
131131

132-
Creates: `spectra.config.json`, `tests/`, `docs/`, `docs/requirements/_requirements.yaml`, `templates/bug-report.md`, `.github/skills/` (6 SKILLs), `.github/agents/` (2 agents), `.vscode/mcp.json`, `.github/workflows/deploy-dashboard.yml`
132+
Creates: `spectra.config.json`, `tests/`, `docs/`, `docs/requirements/_criteria_index.yaml`, `templates/bug-report.md`, `.github/skills/` (9 SKILLs), `.github/agents/` (2 agents), `.vscode/mcp.json`, `.github/workflows/deploy-dashboard.yml`
133133

134134
#### `spectra ai generate`
135135

@@ -181,15 +181,17 @@ Classifies tests as: UP_TO_DATE, OUTDATED, ORPHANED, REDUNDANT.
181181

182182
#### `spectra ai analyze`
183183

184-
Analyze test coverage and extract requirements.
184+
Analyze test coverage and extract acceptance criteria.
185185

186186
```bash
187187
spectra ai analyze --coverage # Three-section report
188188
spectra ai analyze --coverage --auto-link # Write automated_by fields
189189
spectra ai analyze --coverage --format json --output coverage.json # File output
190190
spectra ai analyze --coverage --output-format json # Stdout JSON
191-
spectra ai analyze --extract-requirements # Extract from docs
192-
spectra ai analyze --extract-requirements --dry-run # Preview only
191+
spectra ai analyze --extract-criteria # Extract from docs
192+
spectra ai analyze --extract-criteria --dry-run # Preview only
193+
spectra ai analyze --import-criteria ./criteria.csv # Import from external file
194+
spectra ai analyze --list-criteria # List all criteria
193195
```
194196

195197
#### `spectra validate`
@@ -225,11 +227,13 @@ spectra index --suite checkout # Specific suite
225227

226228
#### `spectra docs index`
227229

228-
Build or refresh the documentation index, then auto-extract requirements.
230+
Build or refresh the documentation index, then auto-extract acceptance criteria.
229231

230232
```bash
231-
spectra docs index # Incremental update + auto-extract requirements
232-
spectra docs index --force # Full rebuild + auto-extract requirements
233+
spectra docs index # Incremental update + auto-extract acceptance criteria
234+
spectra docs index --force # Full rebuild + auto-extract acceptance criteria
235+
spectra docs index --skip-criteria # Index only, skip criteria extraction
236+
spectra docs index --no-interaction --output-format json # SKILL/CI mode
233237
```
234238

235239
#### `spectra dashboard`
@@ -404,7 +408,7 @@ Generated in three formats: JSON, Markdown, HTML. Features: test titles from `_i
404408
"automation_dirs": [],
405409
"scan_patterns": ["**/*.cs", "**/*.ts", "**/*.js", "**/*.py"],
406410
"file_extensions": [".cs", ".ts", ".js", ".py"],
407-
"requirements_file": "docs/requirements/_requirements.yaml"
411+
"criteria_file": "docs/requirements/_criteria_index.yaml"
408412
},
409413
"execution": {
410414
"db_path": ".execution/spectra.db",
@@ -650,6 +654,9 @@ Test areas covered:
650654
| 020 | CLI Non-Interactive | Implemented |
651655
| 021 | Generation Session | Implemented |
652656
| 022 | Bundled SKILLs | Implemented |
657+
| 023 | Criteria Extraction Overhaul | Implemented |
658+
| 023 | Copilot Chat Integration | Implemented |
659+
| 024 | Docs Index SKILL & Coverage Fix | Implemented |
653660

654661
### Spec-Kit Drafts
655662

‎docs/cli-reference.md‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -98,18 +98,22 @@ Reports a summary of updated, unchanged, and skipped files.
9898

9999
### `spectra docs index`
100100

101-
Build or incrementally update the documentation index (`docs/_index.md`), then automatically extract testable requirements.
101+
Build or incrementally update the documentation index (`docs/_index.md`), then automatically extract acceptance criteria.
102102

103103
```bash
104-
spectra docs index # Incremental update + auto-extract requirements
105-
spectra docs index --force # Full rebuild + auto-extract requirements
104+
spectra docs index # Incremental update + auto-extract acceptance criteria
105+
spectra docs index --force # Full rebuild + auto-extract acceptance criteria
106+
spectra docs index --skip-criteria # Index only, skip criteria extraction
107+
spectra docs index --no-interaction --output-format json # SKILL/CI mode
106108
```
107109

108110
The index contains per-document metadata (title, sections with summaries, key entities, word/token counts, content hashes). The AI agent reads this lightweight index (~1-2K tokens) instead of scanning all files.
109111

110112
Content hashes enable incremental updates — only changed files are re-indexed. The index is also auto-refreshed before `spectra ai generate` runs.
111113

112-
After indexing, requirements are automatically extracted from the documentation using the configured AI provider and merged into `_requirements.yaml`. If no provider is configured, the extraction step is skipped.
114+
After indexing, acceptance criteria are automatically extracted from the documentation using the configured AI provider and merged into `_criteria_index.yaml`. If no provider is configured, the extraction step is skipped. Use `--skip-criteria` to skip extraction entirely.
115+
116+
In SKILL/CI mode, the command writes `.spectra-result.json` (structured result) and `.spectra-progress.html` (live progress page with auto-refresh).
113117

114118
See [Document Index](document-index.md) for full details.
115119

‎docs/coverage.md‎

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Coverage Analysis
22

3-
Three-dimensional coverage analysis: Documentation, Requirements, and Automation.
3+
Three-dimensional coverage analysis: Documentation, Acceptance Criteria, and Automation.
44

55
Related: [CLI Reference](cli-reference.md) | [Configuration](configuration.md) | [Test Format](test-format.md)
66

@@ -13,7 +13,7 @@ SPECTRA produces a unified coverage report with three sections:
1313
| Type | What it measures | Data source |
1414
|------|-----------------|-------------|
1515
| **Documentation** | Which docs have linked tests | `source_refs` field in test frontmatter matched against `docs/` |
16-
| **Requirements** | Which requirements are covered | `requirements` field in test frontmatter + `_requirements.yaml` |
16+
| **Acceptance Criteria** | Which criteria are covered | `criteria` field in test frontmatter + `_criteria_index.yaml` |
1717
| **Automation** | Which tests have automation code | `automated_by` field in test frontmatter + code scanning |
1818

1919
## Run Coverage Analysis
@@ -46,14 +46,14 @@ spectra ai analyze --coverage --verbosity detailed
4646
{ "doc": "docs/admin.md", "test_count": 0, "covered": false, "test_ids": [] }
4747
]
4848
},
49-
"requirements_coverage": {
50-
"total_requirements": 5,
51-
"covered_requirements": 3,
49+
"acceptance_criteria_coverage": {
50+
"total": 5,
51+
"covered": 3,
5252
"percentage": 60.00,
53-
"has_requirements_file": true,
53+
"has_criteria_file": true,
5454
"details": [
55-
{ "id": "REQ-042", "title": "Payment rejection", "tests": ["TC-134"], "covered": true },
56-
{ "id": "REQ-043", "title": "Expired card", "tests": [], "covered": false }
55+
{ "id": "AC-042", "title": "Payment rejection", "tests": ["TC-134"], "covered": true },
56+
{ "id": "AC-043", "title": "Expired card", "tests": [], "covered": false }
5757
]
5858
},
5959
"automation_coverage": {
@@ -74,39 +74,39 @@ Measures which documentation files have at least one test referencing them via `
7474

7575
For each doc in `docs/`, SPECTRA checks if any test file has it in its `source_refs` frontmatter field.
7676

77-
## Requirements Coverage
77+
## Acceptance Criteria Coverage
7878

79-
Measures which formal requirements are covered by tests.
79+
Measures which acceptance criteria are covered by tests.
8080

81-
### With `_requirements.yaml`
81+
### With `_criteria_index.yaml`
8282

83-
When a requirements file exists, SPECTRA cross-references the defined requirements with `requirements` fields in test frontmatter. This reveals which requirements have no tests.
83+
When a criteria index file exists, SPECTRA cross-references the defined criteria with `criteria` fields in test frontmatter. This reveals which criteria have no tests.
8484

85-
### Without `_requirements.yaml`
85+
### Without `_criteria_index.yaml`
8686

87-
When no requirements file exists, SPECTRA discovers requirements from test frontmatter only and reports them as a flat list. `has_requirements_file` is `false`.
87+
When no criteria file exists, SPECTRA discovers criteria from test frontmatter only and reports them as a flat list. `has_criteria_file` is `false`.
8888

89-
### Requirements File Format
89+
### Criteria File Format
9090

91-
Create `docs/requirements/_requirements.yaml`:
91+
Create `docs/requirements/_criteria_index.yaml` (or use `spectra ai analyze --extract-criteria` to auto-generate):
9292

9393
```yaml
94-
requirements:
95-
- id: REQ-001
94+
criteria:
95+
- id: AC-001
9696
title: "User can log in with valid credentials"
9797
source: docs/authentication.md
9898
priority: high
99-
- id: REQ-002
99+
- id: AC-002
100100
title: "System rejects invalid passwords"
101101
source: docs/authentication.md
102102
priority: high
103-
- id: REQ-003
103+
- id: AC-003
104104
title: "Admin panel access control"
105105
source: docs/admin.md
106106
priority: high
107107
```
108108
109-
The path to this file is configured via `coverage.requirements_file` in [spectra.config.json](configuration.md).
109+
The path to this file is configured via `coverage.criteria_file` in [spectra.config.json](configuration.md).
110110

111111
## Automation Coverage
112112

@@ -162,7 +162,7 @@ Full coverage settings in `spectra.config.json`:
162162
"automation_dirs": ["tests", "test", "spec", "e2e"],
163163
"scan_patterns": ["[TestCase(\"{id}\")]", "@pytest.mark.manual_test(\"{id}\")"],
164164
"file_extensions": [".cs", ".java", ".py", ".ts"],
165-
"requirements_file": "docs/requirements/_requirements.yaml"
165+
"criteria_file": "docs/requirements/_criteria_index.yaml"
166166
}
167167
}
168168
```
@@ -187,13 +187,13 @@ Three stacked cards — one per coverage type — with:
187187
- Percentage and fill bar (green >= 80%, yellow >= 50%, red < 50%)
188188
- "Show details" toggle that expands a per-item breakdown list
189189
- Documentation: each doc file with test count and covered/uncovered icon
190-
- Requirements: each requirement ID, title, linked test IDs
190+
- Acceptance Criteria: each criterion ID, title, linked test IDs
191191
- Automation: per-suite breakdown (suite name, automated/total, percentage)
192192

193193
### Empty State Guidance
194194

195195
When coverage data is missing or unconfigured, cards show actionable messages:
196-
- **Requirements**: "No requirements tracked yet" with setup instructions
196+
- **Acceptance Criteria**: "No acceptance criteria tracked yet" with setup instructions
197197
- **Automation**: "No automation links detected" with `--auto-link` instructions
198198
- **Documentation**: "All documents have test coverage!" success message when at 100%
199199

0 commit comments

Comments
 (0)
⚡