feat: coverage-aware analysis for generate flow (044)

Surface coverage and criteria context in the analysis phase so the
Copilot agent can propose tests that close real gaps. Adds CoverageSnapshot
builder, context formatter, presenter enrichment, and progress/result
plumbing, plus tests covering the new analysis path.

Author: angelovstanton

CLAUDE.md

@@ -130,3 +130,10 @@ Generated in JSON, Markdown, HTML. Features: test titles from `_index.json`, hum
 
 <!-- MANUAL ADDITIONS START -->
 <!-- MANUAL ADDITIONS END -->
+
+## Active Technologies
+- C# 12, .NET 8+ + GitHub Copilot SDK, System.CommandLine, Spectre.Console, System.Text.Json, YamlDotNet (044-coverage-aware-analysis)
+- File-based (Markdown/YAML/JSON in `test-cases/`, `docs/criteria/`, `_index.json`, `_criteria_index.yaml`) (044-coverage-aware-analysis)
+
+## Recent Changes
+- 044-coverage-aware-analysis: Added C# 12, .NET 8+ + GitHub Copilot SDK, System.CommandLine, Spectre.Console, System.Text.Json, YamlDotNet

Directory.Build.props

@@ -16,7 +16,7 @@
     <EnforceCodeStyleInBuild>true</EnforceCodeStyleInBuild>
 
     <!-- Versioning -->
-    <Version>1.49.0</Version>
+    <Version>1.50.0</Version>
     <Authors>Spectra Contributors</Authors>
     <Company>Spectra</Company>
     <Product>Spectra</Product>

docs/cli-reference.md

@@ -134,7 +134,9 @@ techniques (Equivalence Partitioning, Boundary Value Analysis, Decision
 Table, State Transition, Error Guessing, Use Case). The analysis output
 includes both a category breakdown (`happy_path`, `boundary`, `negative`, …)
 and a technique breakdown (`BVA`, `EP`, `DT`, `ST`, `EG`, `UC`) — see spec
-037. Supports multiple modes.
+037. The analysis step is **coverage-aware** (spec 044): for existing suites,
+it considers test coverage from `_index.json`, acceptance criteria, and
+doc sections to recommend only gap tests. Supports multiple modes.
 
 **Interactive Session** — four-phase guided session:
 

docs/coverage.md

@@ -170,6 +170,20 @@ Scan patterns are templates where `{id}` is replaced with the test ID regex. Exa
 
 If `scan_patterns` is empty, SPECTRA falls back to the legacy `attribute_patterns` regex list.
 
+## Coverage-Aware Generation
+
+When running `spectra ai generate`, the analysis step is coverage-aware for existing suites. Before identifying testable behaviors, the analyzer builds a coverage snapshot from:
+
+- **`_index.json`**: Existing test titles, criteria links, and source refs
+- **`.criteria.yaml` files**: All acceptance criteria, cross-referenced against tests
+- **`docs/_index.md`**: Documentation sections, cross-referenced against test source refs
+
+The AI receives this coverage context and only recommends tests for genuine gaps — uncovered criteria and undocumented sections. For a mature suite with 231 tests covering 38/41 criteria, the analysis recommends ~8 new tests (the actual gap) instead of 139.
+
+For suites with more than 500 tests, the analyzer switches to summary mode to conserve prompt tokens: only coverage statistics and uncovered items are sent, not the full title list.
+
+New suites with no `_index.json` or criteria files work exactly as before — the coverage context is simply omitted.
+
 ## Coverage Configuration
 
 Full coverage settings in `spectra.config.json`:

docs/customization.md

@@ -109,6 +109,12 @@ When analyzing documentation, pay special attention to:
 - Rate limiting and abuse prevention on public APIs
 ```
 
+**Coverage-aware analysis**: The `behavior-analysis.md` template includes a
+`{{coverage_context}}` placeholder (spec 044). When an existing suite has
+coverage data, this resolves to a markdown block listing covered/uncovered
+criteria and doc sections, directing the AI to focus on gaps. If your custom
+template omits this placeholder, coverage context is simply not injected.
+
 **Safe updates**: `spectra update-skills` refreshes unmodified templates to
 the latest version. Modified templates are preserved. Use
 `spectra prompts reset {template}` to restore a template to default.

specs/044-coverage-aware-analysis/checklists/requirements.md

@@ -0,0 +1,35 @@
+# Specification Quality Checklist: Coverage-Aware Behavior Analysis
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2026-04-13
+**Feature**: [spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- All items pass. Spec is ready for `/speckit.plan`.
+- The spec references file formats (`_index.json`, `_criteria_index.yaml`) by name since they are domain concepts (not implementation details) — these are the user-facing data files the feature operates on.

specs/044-coverage-aware-analysis/contracts/coverage-context-contract.md

@@ -0,0 +1,49 @@
+# Contract: Coverage Context in Analysis Output
+
+**Date**: 2026-04-13 | **Feature**: 044-coverage-aware-analysis
+
+## JSON Output Contract (--output-format json)
+
+When `spectra ai generate --analyze-only --output-format json` runs on a suite with existing tests, the `analysis` object in the result JSON includes coverage fields:
+
+### New fields in `analysis` object
+
+```json
+{
+  "status": "success",
+  "analysis": {
+    "total_behaviors": 8,
+    "already_covered": 233,
+    "recommended": 8,
+    "breakdown": { "boundary": 2, "negative": 2, "happy_path": 3, "edge_case": 1 },
+    "technique_breakdown": { "BVA": 3, "EP": 2, "EG": 1, "UC": 2 },
+    "existing_test_count": 231,
+    "total_criteria": 41,
+    "covered_criteria": 38,
+    "uncovered_criteria": 3,
+    "uncovered_criteria_ids": ["AC-039", "AC-040", "AC-041"]
+  }
+}
+```
+
+### Backward compatibility
+
+- All new fields have defaults: `existing_test_count: 0`, `total_criteria: 0`, `covered_criteria: 0`, `uncovered_criteria: 0`, `uncovered_criteria_ids: []`
+- Old consumers that don't read these fields are unaffected
+- When no coverage data is available (new suite), all new fields are 0/empty
+
+## Prompt Template Contract
+
+### New placeholder: `{{coverage_context}}`
+
+Added to `behavior-analysis.md` template. Resolves to a markdown block or empty string.
+
+**Full mode** (tests <= 500): Includes covered criteria IDs, uncovered criteria details, covered/uncovered source refs, and truncated test titles.
+
+**Summary mode** (tests > 500): Includes only coverage statistics and uncovered items. No title list.
+
+**Empty** (no coverage data): Placeholder resolves to empty string. Template behavior is identical to pre-feature.
+
+### User customization
+
+Users can override `behavior-analysis.md` via `.spectra/prompts/behavior-analysis.md`. If their template omits `{{coverage_context}}`, coverage data is simply not injected — no error, no crash.

specs/044-coverage-aware-analysis/data-model.md

@@ -0,0 +1,87 @@
+# Data Model: Coverage-Aware Behavior Analysis
+
+**Date**: 2026-04-13 | **Feature**: 044-coverage-aware-analysis
+
+## New Entities
+
+### CoverageSnapshot
+
+Aggregated view of what's already tested in a suite. Built from three independent data sources.
+
+| Field | Type | Source | Description |
+|-------|------|--------|-------------|
+| ExistingTestCount | int | `_index.json` | Total test cases in the suite |
+| ExistingTestTitles | string[] | `_index.json` | Test titles for dedup (truncated to 80 chars) |
+| CoveredCriteriaIds | HashSet\<string\> | `_index.json` tests → criteria fields | Criteria IDs with at least one linked test |
+| UncoveredCriteria | UncoveredCriterion[] | `_criteria_index.yaml` - CoveredCriteriaIds | Criteria with zero linked tests |
+| CoveredSourceRefs | HashSet\<string\> | `_index.json` tests → source_refs fields | Doc sections with at least one linked test |
+| UncoveredSourceRefs | string[] | `docs/_index.md` - CoveredSourceRefs | Doc sections with no linked tests |
+| TotalCriteriaCount | int | `_criteria_index.yaml` | Total criteria across all sources |
+| Mode | Full \| Summary | Computed | Full if tests <= 500, Summary otherwise |
+
+**Lifecycle**: Created per analysis run. Not persisted — derived from committed files.
+
+**Validation**:
+- Empty/missing data sources produce zero values (not errors)
+- CoveredCriteriaIds uses case-insensitive comparison
+- CoveredSourceRefs uses case-insensitive comparison
+
+### UncoveredCriterion
+
+A single acceptance criterion that has no linked tests.
+
+| Field | Type | Source | Description |
+|-------|------|--------|-------------|
+| Id | string | `.criteria.yaml` | Criterion identifier (e.g., "AC-001") |
+| Text | string | `.criteria.yaml` | Full criterion text |
+| Source | string? | `.criteria.yaml` | Source document path |
+| Priority | string | `.criteria.yaml` | Priority level (high/medium/low) |
+
+## Modified Entities
+
+### GenerateAnalysis (in GenerateResult.cs)
+
+New fields added alongside existing ones:
+
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| ExistingTestCount | int | 0 | Total tests in suite (from snapshot) |
+| TotalCriteria | int | 0 | Total acceptance criteria count |
+| CoveredCriteria | int | 0 | Criteria with linked tests |
+| UncoveredCriteria | int | 0 | Criteria without linked tests |
+| UncoveredCriteriaIds | string[] | [] | IDs of uncovered criteria |
+
+**Backward compatibility**: All new fields default to 0 or empty. Old JSON consumers ignore unknown fields. Old JSON without new fields deserializes with defaults.
+
+### BehaviorAnalysisResult (unchanged schema)
+
+No structural changes. The `AlreadyCovered` field now uses the snapshot count (when available) instead of title-similarity heuristic. `RecommendedCount` derives from `TotalBehaviors - AlreadyCovered` as before.
+
+## Data Flow
+
+```
+_index.json ──────┐
+                   │
+.criteria.yaml ────┤──→ CoverageSnapshotBuilder.BuildAsync() ──→ CoverageSnapshot
+                   │
+docs/_index.md ────┘
+                                    │
+                                    ▼
+                   CoverageContextFormatter.Format(snapshot) ──→ string (markdown block)
+                                    │
+                                    ▼
+                   PlaceholderResolver.Resolve(template, {coverage_context: block})
+                                    │
+                                    ▼
+                   BehaviorAnalyzer.AnalyzeAsync() ──→ BehaviorAnalysisResult
+                                    │
+                                    ▼
+                   GenerateAnalysis (enriched with snapshot stats)
+```
+
+## Relationships to Existing Models
+
+- **TestIndexEntry** (`_index.json`): Read `Criteria` (string[]) and `SourceRefs` (string[]) fields
+- **AcceptanceCriterion** (`.criteria.yaml`): Read `Id`, `Text`, `Source`/`SourceDoc`, `Priority` fields
+- **DocumentIndexEntry** (`docs/_index.md`): Read `Path` and `Sections[].Heading` for source ref matching
+- **CriteriaIndex** (`_criteria_index.yaml`): Read `Sources[]` to locate `.criteria.yaml` files

specs/044-coverage-aware-analysis/plan.md

@@ -0,0 +1,100 @@
+# Implementation Plan: Coverage-Aware Behavior Analysis
+
+**Branch**: `044-coverage-aware-analysis` | **Date**: 2026-04-13 | **Spec**: [spec.md](spec.md)
+**Input**: Feature specification from `/specs/044-coverage-aware-analysis/spec.md`
+
+## Summary
+
+The behavior analysis step in `spectra ai generate` currently treats documentation as a blank slate, identifying all testable behaviors and applying a weak title-similarity heuristic for deduplication. For mature suites this produces wildly inaccurate recommendations (139 new tests when 8 are actually needed). This plan adds a coverage snapshot that feeds existing test index, criteria coverage, and doc section coverage into the analysis prompt so the AI only recommends tests for genuine gaps.
+
+## Technical Context
+
+**Language/Version**: C# 12, .NET 8+  
+**Primary Dependencies**: GitHub Copilot SDK, System.CommandLine, Spectre.Console, System.Text.Json, YamlDotNet  
+**Storage**: File-based (Markdown/YAML/JSON in `test-cases/`, `docs/criteria/`, `_index.json`, `_criteria_index.yaml`)  
+**Testing**: xUnit (~1279 tests across 3 test projects)  
+**Target Platform**: Windows/macOS/Linux CLI  
+**Project Type**: CLI tool with MCP server  
+**Performance Goals**: Coverage snapshot computation < 2s for 500 tests  
+**Constraints**: Prompt token overhead < 5,000 tokens; switch to summary mode > 500 tests  
+**Scale/Scope**: Suites up to 500+ tests, 50+ criteria, 20+ doc sections
+
+## Constitution Check
+
+*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
+
+| Principle | Status | Notes |
+|-----------|--------|-------|
+| I. GitHub as Source of Truth | PASS | All data read from committed files (`_index.json`, `.criteria.yaml`, `_index.md`) |
+| II. Deterministic Execution | PASS | Same coverage data + same docs = same snapshot; no randomness in snapshot building |
+| III. Orchestrator-Agnostic Design | PASS | Coverage context is injected into the prompt template, not tied to a specific LLM |
+| IV. CLI-First Interface | PASS | Feature is CLI-only; no new UI required; existing flags preserved |
+| V. Simplicity (YAGNI) | PASS | One new model + one builder service + prompt template changes. No new abstractions beyond what's needed. Reuses existing `IndexWriter`, `CriteriaFileReader`, `DocumentIndexReader` |
+
+**Quality Gates**: All existing gates preserved. No new validation rules needed.
+
+## Project Structure
+
+### Documentation (this feature)
+
+```text
+specs/044-coverage-aware-analysis/
+├── plan.md              # This file
+├── research.md          # Phase 0 output
+├── data-model.md        # Phase 1 output
+├── contracts/           # Phase 1 output
+│   └── coverage-context-contract.md
+├── quickstart.md        # Phase 1 output
+└── tasks.md             # Phase 2 output (/speckit.tasks)
+```
+
+### Source Code (repository root)
+
+```text
+src/
+  Spectra.CLI/
+    Agent/
+      Analysis/
+        BehaviorAnalysisResult.cs      # Existing
+        IdentifiedBehavior.cs          # Existing
+        CoverageSnapshot.cs            # NEW: model + UncoveredCriterion record
+        CoverageSnapshotBuilder.cs     # NEW: builds snapshot from index/criteria/docs
+        CoverageContextFormatter.cs    # NEW: formats snapshot for prompt injection
+      Copilot/
+        BehaviorAnalyzer.cs            # MODIFIED: accept snapshot, inject context
+    Commands/
+      Generate/
+        GenerateHandler.cs             # MODIFIED: build snapshot before analysis
+    Output/
+        AnalysisPresenter.cs           # MODIFIED: show coverage summary
+    Progress/
+        ProgressPageWriter.cs          # MODIFIED: coverage snapshot in HTML
+    Prompts/
+      Content/
+        behavior-analysis.md           # MODIFIED: add {{coverage_context}} placeholder
+    Results/
+        GenerateResult.cs              # MODIFIED: add coverage fields to GenerateAnalysis
+
+  Spectra.Core/
+    # No changes — read existing data via existing readers
+
+tests/
+  Spectra.CLI.Tests/
+    Agent/
+      Analysis/
+        CoverageSnapshotBuilderTests.cs  # NEW: 8 tests
+        CoverageContextFormatterTests.cs # NEW: 5 tests
+    Commands/
+      Generate/
+        GenerateHandlerCoverageTests.cs  # NEW: 2 integration tests
+    Output/
+        AnalysisPresenterCoverageTests.cs # NEW: 3 tests
+    Results/
+        GenerateAnalysisCoverageTests.cs  # NEW: 3 tests
+```
+
+**Structure Decision**: All new files go within the existing `Agent/Analysis/` directory (model + builder + formatter). No new project or directory structure needed — follows existing patterns exactly.
+
+## Complexity Tracking
+
+> No violations — all gates pass. No complexity justification needed.

specs/044-coverage-aware-analysis/quickstart.md

@@ -0,0 +1,46 @@
+# Quickstart: Coverage-Aware Behavior Analysis
+
+**Date**: 2026-04-13 | **Feature**: 044-coverage-aware-analysis
+
+## What changed
+
+The `spectra ai generate` analysis step now considers existing test coverage before recommending new tests. For mature suites, this means accurate gap-only recommendations instead of inflated counts.
+
+## How to verify
+
+### 1. Run analysis on an existing suite
+
+```bash
+spectra ai generate --suite my-suite --analyze-only
+```
+
+**Expected**: Output shows a coverage snapshot with criteria/doc coverage ratios and recommends only tests for genuine gaps.
+
+### 2. Check JSON output
+
+```bash
+spectra ai generate --suite my-suite --analyze-only --output-format json
+```
+
+**Expected**: JSON includes `existing_test_count`, `total_criteria`, `covered_criteria`, `uncovered_criteria`, `uncovered_criteria_ids` fields.
+
+### 3. Verify graceful degradation
+
+```bash
+spectra ai generate --suite new-suite --analyze-only
+```
+
+**Expected**: New suite with no `_index.json` produces standard analysis (no coverage section, no errors).
+
+## Key files to review
+
+| File | Change |
+|------|--------|
+| `src/Spectra.CLI/Agent/Analysis/CoverageSnapshot.cs` | New model |
+| `src/Spectra.CLI/Agent/Analysis/CoverageSnapshotBuilder.cs` | New builder |
+| `src/Spectra.CLI/Agent/Analysis/CoverageContextFormatter.cs` | New formatter |
+| `src/Spectra.CLI/Agent/Copilot/BehaviorAnalyzer.cs` | Accepts snapshot, injects context |
+| `src/Spectra.CLI/Commands/Generate/GenerateHandler.cs` | Builds snapshot before analysis |
+| `src/Spectra.CLI/Prompts/Content/behavior-analysis.md` | New `{{coverage_context}}` placeholder |
+| `src/Spectra.CLI/Output/AnalysisPresenter.cs` | Coverage summary display |
+| `src/Spectra.CLI/Results/GenerateResult.cs` | New fields on GenerateAnalysis |