Merge branch '039-unify-critic-providers'

Author: angelovstanton

CLAUDE.md

@@ -209,6 +209,7 @@ spectra config list-automation-dirs                 # List dirs with existence s
 - **Tests:** xUnit with structured results (never throw on validation errors)
 
 ## Recent Changes
+- 039-unify-critic-providers: ✅ COMPLETE - Aligned the critic provider validation set with the generator provider list. `CriticFactory.SupportedProviders` now contains exactly the canonical 5 (`github-models`, `azure-openai`, `azure-anthropic`, `openai`, `anthropic`) — same as `ai.providers[].name`. Removed `azure-deepseek` (not in the generator set) and `google` (Copilot SDK cannot route to it). New private `LegacyAliases` dictionary maps `github` → `github-models` (soft alias with one-line stderr deprecation warning); new private `HardErrorProviders` set contains `google` and produces a hard validation error listing all 5 supported providers. New `internal static ResolveProvider(string?)` helper centralizes the lookup; called from both `TryCreate` and `TryCreateAsync` (in the async path BEFORE the Copilot availability check so unknown providers fail fast). New public `DefaultProvider = "github-models"` constant; empty/whitespace input falls back to it. Case-insensitive matching with lowercase normalization. `IsSupported` returns true for canonical names AND legacy `github`, false for `google`/unknown. `CriticConfig.Provider` XML doc updated to list the canonical 5; `GetEffectiveModel()` and `GetDefaultApiKeyEnv()` switch statements gain explicit cases for `github-models`, `azure-openai`, `azure-anthropic` (defaults: `gpt-4o-mini`/`gpt-4o-mini`/`claude-3-5-haiku-latest` and `GITHUB_TOKEN`/`AZURE_OPENAI_API_KEY`/`AZURE_ANTHROPIC_API_KEY`); legacy `github`/`google` cases retained for read-side safety. Existing `CriticFactoryTests`, `CriticAuthTests`, `VerificationIntegrationTests`, `CriticConfigTests` updated to drop `google` from "supported" assertions and add `azure-openai`/`azure-anthropic` to the canonical theory data. Docs updated: `docs/configuration.md` lists the canonical 5 with two new examples (Azure-only billing, cross-provider critic); `docs/grounding-verification.md` example uses `github-models` instead of `google`, supported list updated, legacy-values note added. New `CriticFactoryProviderTests.cs` with 8 tests (azure-openai accept, azure-anthropic accept, case-insensitive normalization, github→github-models alias with stderr warning, google hard error with 5-provider listing, unknown provider error, empty fallback, whitespace fallback). Backward compatible: all existing configurations on `openai`/`anthropic`/`github-models` continue to work unchanged. 1551 total tests passing (491 Core + 709 CLI + 351 MCP).
 - 038-testimize-integration: ✅ COMPLETE - Optional integration with the external Testimize.MCP.Server tool for algorithmic test data optimization (BVA, EP, pairwise, ABC). Disabled by default; zero impact on users who do not opt in. New `testimize` section in `spectra.config.json` (`enabled` defaults to `false`, `mode`, `strategy`, optional `settings_file`, `mcp.command`/`args`, optional `abc_settings`). New models in `src/Spectra.Core/Models/Config/TestimizeConfig.cs` (`TestimizeConfig`, `TestimizeMcpConfig`, `TestimizeAbcSettings`). New `SpectraConfig.Testimize` property defaulting to `new()`. New `src/Spectra.CLI/Agent/Testimize/TestimizeMcpClient.cs` — `IAsyncDisposable` wrapper around the Testimize child process with stdio JSON-RPC framing (`Content-Length` header), `StartAsync` (returns false on missing tool, never throws), `IsHealthyAsync` (5s budget), `CallToolAsync` (30s budget, returns null on any failure), idempotent `DisposeAsync` (kills child process tree). New `src/Spectra.CLI/Agent/Testimize/TestimizeTools.cs` with two `AIFunction` factories: `CreateGenerateTestDataTool` (forwards to Testimize MCP `generate_hybrid_test_cases`/`generate_pairwise_test_cases`/`generate_combinatorial_test_cases` based on strategy) and `CreateAnalyzeFieldSpecTool` (local heuristic extractor for "X to Y characters", "between X and Y", "valid email", "required field"). Tool description explicitly mentions "boundary values", "equivalence classes", and "pairwise" so the model knows when to call them. New `TestimizeDetector.IsInstalledAsync` shells out to `dotnet tool list -g` (5s timeout, never throws). `CopilotGenerationAgent.GenerateTestsAsync` conditionally registers the Testimize tools after the existing 7 tools when `_config.Testimize.Enabled` and the MCP client passes a health probe; the client is disposed in a `finally` block on every exit path (success, exception, cancellation). `BehaviorAnalyzer.BuildAnalysisPrompt` and `GenerationAgent.BuildFullPrompt` populate a new `testimize_enabled` placeholder (`"true"` or `""`). `behavior-analysis.md` and `test-generation.md` templates gain `{{#if testimize_enabled}}` blocks with technique-aware instructions. New `spectra testimize check` CLI command (`src/Spectra.CLI/Commands/Testimize/TestimizeCheckHandler.cs` + `TestimizeCommand.cs`) reports enabled/installed/healthy/mode/strategy/settings status, supports `--output-format json`, never starts the MCP process when disabled (FR-028), includes install instruction when not installed. New `TestimizeCheckResult` typed result. Embedded `Templates/spectra.config.json` updated to include the `testimize` section with `enabled: false` so `spectra init` writes it by default. Existing `spectra init` already refuses to overwrite an existing config without `--force`, so re-init preservation is automatic. Graceful degradation everywhere: tool not installed → warning + AI fallback; process crashes mid-generation → catch + AI fallback; server returns null/empty/garbage → AI fallback; per-call 30s timeout → AI fallback. Exit code 0 in all degradation paths. New tests: `TestimizeConfigTests` (6), `TestimizeMcpClientTests` (4), `TestimizeMcpClientGracefulTests` (4), `TestimizeToolsTests` (2), `AnalyzeFieldSpecTests` (7), `TestimizeDetectorTests` (1), `TestimizeConditionalBlockTests` (4), `TestimizeCheckHandlerTests` (3) — 31 net new tests (488 Core + 699 CLI + 351 MCP = 1538 total passing).
 - 037-istqb-test-techniques: ✅ COMPLETE - ISTQB test design techniques in prompt templates. All five built-in prompt templates (`src/Spectra.CLI/Prompts/Content/*.md`) updated to teach the AI six ISTQB black-box techniques: Equivalence Partitioning (EP), Boundary Value Analysis (BVA), Decision Table (DT), State Transition (ST), Error Guessing (EG), Use Case (UC). `behavior-analysis.md` rewritten with six TECHNIQUE sections, distribution guidelines (40%-of-any-category cap, ≥4 BVA per range, mandatory invalid state transitions), and a new `technique` field in the JSON output schema. `test-generation.md` adds a "TEST DESIGN TECHNIQUE RULES" section with WRONG/RIGHT examples for BVA exact values, EP class naming, DT condition values, ST starting/action/resulting state, and EG concrete error scenarios. `test-update.md` adds a "Technique Completeness Check" so updates flag tests as OUTDATED when docs introduce new ranges/rules/states. `critic-verification.md` adds a "Technique Verification" section returning PARTIAL for BVA boundary mismatches and HALLUCINATED for DT conditions not in docs. `criteria-extraction.md` adds optional `technique_hint` per criterion. New `IdentifiedBehavior.Technique` string field (default `""` for backward compatibility), new `BehaviorAnalysisResult.TechniqueBreakdown` map (excludes empty techniques), new `AcceptanceCriterion.TechniqueHint` `string?` (omitted from YAML when null). `BehaviorAnalyzer.BuildAnalysisPrompt` legacy fallback updated with condensed ISTQB instructions. `GenerateAnalysis.TechniqueBreakdown` always serialized as `{}` when empty for stable SKILL/CI contract. `AnalysisPresenter` and `ProgressPageWriter` render a "Technique Breakdown" section beneath the existing category breakdown in fixed display order BVA, EP, DT, ST, EG, UC. `CriteriaExtractor` parses and normalizes the technique hint; `LoadCriteriaContextAsync` forwards `technique_hint` into the generation prompt. `BuildPrompt` reinforces ISTQB technique application in the per-batch user prompt. `spectra-generate` SKILL shows both category and technique breakdowns from analysis JSON. `spectra-help` SKILL gains an "ISTQB Test Design Techniques" section pointing users to `spectra prompts reset --all`. `spectra-quickstart` SKILL flagged as updated with the new techniques. `spectra-generation` agent prompt notes the technique breakdown. Existing user-customized `.spectra/prompts/*.md` files are NOT auto-overwritten (spec 022 hash-tracking preserved); users opt in via `spectra prompts reset` or `spectra update-skills`. New tests: `BehaviorAnalysisTemplateTests` (6), `BehaviorAnalyzerTechniqueTests` (3), `BehaviorAnalyzerLegacyFallbackTests` (3), `BehaviorAnalysisResultTechniqueTests` (2), `GenerateResultTechniqueBreakdownTests` (3), `ProgressPageTechniqueBreakdownTests` (3), `TestGenerationTemplateTests` (6), `PromptTemplateMigrationTests` (2), `TestUpdateTemplateTests` (5), `CriticVerificationTemplateTests` (5), `CriteriaExtractionTemplateTests` (6), `AcceptanceCriterionTechniqueHintTests` (4). One existing test (`BuildPrompt_WithoutLoader_UsesLegacy`) updated to assert on the new `boundary` and `error_handling` categories (replacing the dropped `performance` assertion). 48 new tests. 1507 total tests passing (482 Core + 674 CLI + 351 MCP).
 - 034-github-pages-docs: ✅ COMPLETE - GitHub Pages documentation site. Deployed existing `docs/` markdown files as a branded documentation site using Just the Docs Jekyll theme at `automatetheplanet.github.io/Spectra/`. Custom `spectra` color scheme matching ATP brand identity (dark navy sidebar #16213e, teal accents #00bcd4, white content area). Sidebar navigation with hierarchical grouping (User Guide, Architecture, Execution Agents, Deployment sections). Built-in client-side search across all pages. Landing page with quick install and feature overview. New `.github/workflows/docs.yml` (jekyll-build-pages → deploy-pages) auto-deploys on push to `main` when `docs/**` changes; one-time manual setting required (Settings → Pages → Source: GitHub Actions). "Edit this page" links on every page via `gh_edit_*` config. Excluded machine-generated files (`_index.md`, `_index.yaml`, `_index.json`, `criteria/`, `**/*.criteria.yaml`, `_criteria_index.yaml`) from site build. Added YAML frontmatter (`title`, `nav_order`, `parent`) to all 20 existing user-facing doc files in their actual subfolder locations — no files moved. Created 8 new files: `docs/_config.yml`, `docs/Gemfile`, `docs/_sass/color_schemes/spectra.scss`, `docs/index.md`, `docs/user-guide.md`, `docs/architecture.md`, `docs/execution-agents.md`, `docs/deployment.md`. Updated `README.md` with docs site badge and pointer. Documentation-only change — no C# code, no test changes.

docs/configuration.md

@@ -114,12 +114,47 @@ Grounding verification configuration. See [Grounding Verification](grounding-ver
 | Property | Type | Default | Description |
 |----------|------|---------|-------------|
 | `enabled` | bool | `false` | Enable dual-model verification |
-| `provider` | string | — | Critic provider (`google`, `openai`, `anthropic`, `github`) |
+| `provider` | string | `github-models` | Critic provider — same set as the generator: `github-models`, `azure-openai`, `azure-anthropic`, `openai`, `anthropic`. The legacy value `github` is still recognized as an alias for `github-models` (deprecation warning). |
 | `model` | string | — | Critic model identifier |
 | `api_key_env` | string | — | Environment variable for critic API key |
 | `base_url` | string | — | Custom API endpoint |
 | `timeout_seconds` | int | `30` | Critic request timeout |
 
+#### Example: Azure-only billing (generator + critic on the same account)
+
+```json
+{
+  "ai": {
+    "providers": [
+      { "name": "azure-openai", "model": "gpt-4.1-mini", "enabled": true }
+    ],
+    "critic": {
+      "enabled": true,
+      "provider": "azure-openai",
+      "model": "gpt-4o"
+    }
+  }
+}
+```
+
+#### Example: Cross-provider critic for independent verification
+
+```json
+{
+  "ai": {
+    "providers": [
+      { "name": "azure-openai", "model": "gpt-4.1-mini", "enabled": true }
+    ],
+    "critic": {
+      "enabled": true,
+      "provider": "anthropic",
+      "model": "claude-3-5-haiku-latest",
+      "api_key_env": "ANTHROPIC_API_KEY"
+    }
+  }
+}
+```
+
 ## `generation` — Test Generation Settings
 
 | Property | Type | Default | Description |

docs/grounding-verification.md

@@ -88,21 +88,29 @@ Configure the critic in `spectra.config.json`:
   "ai": {
     "critic": {
       "enabled": true,
-      "provider": "google",
-      "model": "gemini-2.0-flash",
+      "provider": "github-models",
+      "model": "gpt-4o-mini",
       "timeout_seconds": 30
     }
   }
 }
 ```
 
-Supported critic providers: `google`, `openai`, `anthropic`, `github`
+Supported critic providers (spec 039 — same set as the generator):
+`github-models`, `azure-openai`, `azure-anthropic`, `openai`, `anthropic`.
 
 Default API key environment variables:
-- Google: `GOOGLE_API_KEY`
-- OpenAI: `OPENAI_API_KEY`
-- Anthropic: `ANTHROPIC_API_KEY`
-- GitHub: `GITHUB_TOKEN`
+- `github-models`: `GITHUB_TOKEN`
+- `azure-openai`: `AZURE_OPENAI_API_KEY`
+- `azure-anthropic`: `AZURE_ANTHROPIC_API_KEY`
+- `openai`: `OPENAI_API_KEY`
+- `anthropic`: `ANTHROPIC_API_KEY`
+
+> **Legacy values**: `provider: "github"` is accepted as a soft alias for
+> `github-models` (with a one-line deprecation warning on stderr). The legacy
+> value `provider: "google"` is no longer supported — the Copilot SDK runtime
+> cannot route to Google. Update your config to one of the canonical five
+> providers above.
 
 ## Skip Verification
 

specs/039-unify-critic-providers/checklists/requirements.md

@@ -0,0 +1,36 @@
+# Specification Quality Checklist: Unified Critic Provider List
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2026-04-11
+**Feature**: [spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- 3 user stories: P1 (Azure-only billing), P2 (legacy compat), P2 (docs alignment).
+- Branch number 039 (auto-allocated; source doc was labelled 025).
+- Small alignment fix — no architectural change.

specs/039-unify-critic-providers/contracts/critic-provider.schema.json

@@ -0,0 +1,20 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://spectra.local/contracts/critic-provider.schema.json",
+  "title": "CriticConfig.provider field",
+  "description": "Schema for ai.critic.provider in spectra.config.json after spec 039. The five canonical names are accepted (case-insensitive). Legacy 'github' is a soft alias for 'github-models'. Legacy 'google' is rejected.",
+  "type": "string",
+  "anyOf": [
+    {
+      "enum": ["github-models", "azure-openai", "azure-anthropic", "openai", "anthropic"],
+      "description": "Canonical provider names — same set as the generator."
+    },
+    {
+      "const": "github",
+      "description": "Legacy alias — accepted, rewritten to 'github-models', stderr warning emitted."
+    }
+  ],
+  "not": {
+    "const": "google"
+  }
+}

specs/039-unify-critic-providers/data-model.md

@@ -0,0 +1,53 @@
+# Data Model: Unified Critic Provider List
+
+**Feature**: 039-unify-critic-providers
+**Date**: 2026-04-11
+
+## Overview
+
+No new entities. Two existing files change semantically.
+
+## Entities
+
+### CriticConfig (modified — docstring + switch defaults only)
+
+**File**: `src/Spectra.Core/Models/Config/CriticConfig.cs`
+
+The on-disk JSON schema is unchanged. What changes:
+
+| Field | Change |
+|-------|--------|
+| `Provider` | Docstring updated from `"google", "openai", "anthropic", "github"` to `"github-models", "azure-openai", "azure-anthropic", "openai", "anthropic"` (with note that legacy `github` and `google` are still recognized). |
+| `GetEffectiveModel()` | Switch statement gains explicit cases for `github-models`, `azure-openai`, `azure-anthropic` mapping to the same defaults as the generator. Legacy cases (`google`, `github`) retained for read-side safety. |
+| `GetDefaultApiKeyEnv()` | Same shape — gains canonical cases, legacy cases retained. |
+
+### CriticFactory (modified)
+
+**File**: `src/Spectra.CLI/Agent/Critic/CriticFactory.cs`
+
+| Member | Change |
+|--------|--------|
+| `SupportedProviders` | Reduced from 7 entries to the canonical 5: `github-models`, `azure-openai`, `azure-anthropic`, `openai`, `anthropic`. (Removes `azure-deepseek`, `google`.) |
+| `LegacyAliases` (NEW, private static) | Map: `"github" → "github-models"` (soft, with deprecation warning). |
+| `HardErrorProviders` (NEW, private static) | Set: `{ "google" }`. |
+| `TryCreate(CriticConfig)` | Now validates `config.Provider` against the canonical set + legacy aliases. Returns `Failed(...)` for unknown or hard-error providers. Emits a one-line stderr deprecation warning when an alias is used. Uses the resolved/normalized provider name when constructing `CopilotCritic`. |
+| `TryCreateAsync(CriticConfig, CancellationToken)` | Same validation as `TryCreate`, applied before the Copilot availability check. |
+| `IsSupported(string)` | Updated to also accept legacy aliases (returns true for `github`). Returns false for `google` and unknown values. |
+
+## State Transitions
+
+None.
+
+## Validation Rules
+
+| Rule | Source | Enforcement |
+|------|--------|-------------|
+| Provider name in canonical 5 (or alias) | spec FR-001, FR-002 | Hard: `CriticFactory.TryCreate` returns `Failed` otherwise. |
+| Case-insensitive | spec FR-002 | Hard: lowercase before lookup. |
+| `google` → hard error | spec FR-006 | Hard: `Failed` with explicit error message listing supported providers. |
+| `github` → soft alias to `github-models` | spec FR-005 | Soft: rewrite + stderr warning. |
+| Empty/null provider when enabled | spec FR-004 | Soft: fall back to default `github-models`. |
+
+## Migration
+
+No on-disk migration. Config files are unchanged. Users with `provider: "github"` see a deprecation warning on next run; users with `provider: "google"` see a hard error directing them to update.