Implement 015-auto-requirements-extraction

AI-powered extraction of testable requirements from documentation.
New RequirementsWriter handles YAML merge, duplicate detection
(normalized title + substring matching), sequential ID allocation
(REQ-NNN), and atomic file writes. RequirementsExtractor uses
Copilot SDK with RFC 2119 priority inference.

CLI: spectra ai analyze --extract-requirements [--dry-run]

Author: angelovstanton

CLAUDE.md

@@ -135,6 +135,8 @@ spectra ai analyze --coverage --format json --output coverage.json
 spectra ai analyze --coverage --format markdown --output coverage.md
 spectra ai analyze --coverage --auto-link                        # Write automated_by back into test files
 spectra ai analyze --coverage --verbosity detailed
+spectra ai analyze --extract-requirements            # Extract requirements from docs
+spectra ai analyze --extract-requirements --dry-run  # Preview without writing
 
 # Automation Directory Management (013-cli-ux-improvements)
 spectra config add-automation-dir ../new-tests     # Add automation dir for coverage
@@ -151,6 +153,7 @@ spectra config list-automation-dirs                 # List dirs with existence s
 - **Tests:** xUnit with structured results (never throw on validation errors)
 
 ## Recent Changes
+- 015-auto-requirements-extraction: ✅ COMPLETE - AI-powered extraction of testable requirements from documentation. New models: `ExtractionResult`, `DuplicateMatch`. New service: `RequirementsWriter` handles YAML merge, duplicate detection (normalized title + substring matching), sequential ID allocation (REQ-NNN, never reuse gaps), atomic writes. New `RequirementsExtractor` uses Copilot SDK to extract requirements with RFC 2119 priority inference. CLI: `spectra ai analyze --extract-requirements [--dry-run]`. 11 new RequirementsWriter tests.
 - 014-open-source-ready: ✅ COMPLETE - Open source readiness. README redesign with banner placeholder, shields.io badges, value props, feature showcase, quickstart. CI pipeline (`.github/workflows/ci.yml`) — build+test on push/PR. NuGet publish pipeline (`.github/workflows/publish.yml`) — tag-triggered pack+push for Spectra.CLI and Spectra.MCP. All 1071 tests passing (fixed parallel test isolation with `[Collection("WorkingDirectory")]`). GitHub issue templates (bug report, feature request), PR template, Dependabot config. All README doc links verified.
 - 013-cli-ux-improvements: ✅ COMPLETE - CLI UX improvements for discoverability and workflow. New `NextStepHints` helper prints context-aware next-step suggestions after every command (init, generate, analyze, dashboard, validate, docs index, index) in dimmed text, suppressed by `--quiet` or piped output. Init flow: new interactive prompts for automation directory setup (`coverage.automation_dirs`) and critic model configuration (`ai.critic`). New config subcommands: `spectra config add-automation-dir`, `remove-automation-dir`, `list-automation-dirs`. Interactive generation mode: continuation menu after suite completion (generate more, switch suite, create suite, exit) with session summary. 18 new tests.
 - 012-dashboard-branding: ✅ COMPLETE - Dashboard branding and theming customization. New models: `BrandingConfig`, `ColorPaletteConfig` in `DashboardConfig`. New service: `BrandingInjector` handles company name, logo, favicon, CSS variable overrides, dark theme, and custom CSS injection via template placeholders. `SampleDataFactory` provides mock data for `--preview` mode. Light/dark theme presets via CSS custom properties. Config: `dashboard.branding` section in spectra.config.json with `company_name`, `logo`, `favicon`, `theme`, `colors`, `custom_css`. CLI: `spectra dashboard --preview` for branding verification. 51 new tests (8 config + 21 injector + 9 sample data + 13 existing generator).

specs/015-auto-requirements-extraction/tasks.md

@@ -0,0 +1,168 @@
+using System.Text.Json;
+using GitHub.Copilot.SDK;
+using Spectra.Core.Models.Coverage;
+using SpectraProviderConfig = Spectra.Core.Models.Config.ProviderConfig;
+
+namespace Spectra.CLI.Agent.Copilot;
+
+/// <summary>
+/// Extracts testable requirements from documentation using the Copilot SDK.
+/// </summary>
+public sealed class RequirementsExtractor
+{
+    private readonly SpectraProviderConfig _provider;
+    private readonly Action<string>? _onStatus;
+
+    private readonly string _basePath;
+
+    public RequirementsExtractor(SpectraProviderConfig provider, string basePath, Action<string>? onStatus = null)
+    {
+        _provider = provider;
+        _basePath = basePath;
+        _onStatus = onStatus;
+    }
+
+    /// <summary>
+    /// Extracts testable requirements from source documents.
+    /// </summary>
+    public async Task<IReadOnlyList<RequirementDefinition>> ExtractAsync(
+        IReadOnlyList<Spectra.Core.Models.DocumentEntry> documents,
+        IReadOnlyList<RequirementDefinition> existingRequirements,
+        CancellationToken ct = default)
+    {
+        if (documents.Count == 0)
+            return [];
+
+        try
+        {
+            var service = await CopilotService.GetInstanceAsync(ct);
+
+            _onStatus?.Invoke("Creating extraction session...");
+            await using var session = await service.CreateGenerationSessionAsync(
+                _provider,
+                ct: ct);
+
+            var prompt = await BuildExtractionPromptAsync(documents, existingRequirements, ct);
+
+            _onStatus?.Invoke("Extracting requirements from documentation...");
+            var response = await session.SendAndWaitAsync(
+                new MessageOptions { Prompt = prompt },
+                timeout: TimeSpan.FromMinutes(3),
+                cancellationToken: ct);
+
+            var responseText = response?.Data?.Content ?? "";
+            return ParseResponse(responseText);
+        }
+        catch (Exception ex)
+        {
+            _onStatus?.Invoke($"Extraction failed: {ex.Message}");
+            return [];
+        }
+    }
+
+    private async Task<string> BuildExtractionPromptAsync(
+        IReadOnlyList<Spectra.Core.Models.DocumentEntry> documents,
+        IReadOnlyList<RequirementDefinition> existing,
+        CancellationToken ct)
+    {
+        var docParts = new List<string>();
+        foreach (var doc in documents)
+        {
+            var fullPath = Path.Combine(_basePath, doc.Path);
+            var content = File.Exists(fullPath)
+                ? await File.ReadAllTextAsync(fullPath, ct)
+                : doc.Preview;
+            docParts.Add($"## Document: {doc.Path}\n\n{content}");
+        }
+        var docContent = string.Join("\n\n---\n\n", docParts);
+
+        var existingTitles = existing.Count > 0
+            ? "Already extracted requirements (DO NOT duplicate these):\n" +
+              string.Join("\n", existing.Select(r => $"- {r.Id}: {r.Title}"))
+            : "No existing requirements.";
+
+        return $$"""
+            You are a requirements analyst. Extract all testable behavioral requirements from the documentation below.
+
+            For each requirement:
+            - title: A concise statement of the testable behavior (e.g., "System locks account after 5 failed login attempts")
+            - source: The document path where this behavior is described
+            - priority: Based on RFC 2119 language:
+              - "high" if the document uses MUST, SHALL, REQUIRED, CRITICAL, or similar mandatory language
+              - "medium" if the document uses SHOULD, RECOMMENDED, or similar advisory language
+              - "low" if the document uses MAY, OPTIONAL, NICE TO HAVE, or similar permissive language
+              - Default to "medium" if no clear priority language is present
+
+            {{existingTitles}}
+
+            Respond ONLY with a JSON array. No markdown, no explanation. Example:
+            [
+              {"title": "User can reset password via email", "source": "docs/auth.md", "priority": "high"},
+              {"title": "System displays warning on weak password", "source": "docs/auth.md", "priority": "medium"}
+            ]
+
+            Documentation:
+            {{docContent}}
+            """;
+    }
+
+    private static IReadOnlyList<RequirementDefinition> ParseResponse(string responseText)
+    {
+        if (string.IsNullOrWhiteSpace(responseText))
+            return [];
+
+        try
+        {
+            // Extract JSON array from response (may have markdown code fences)
+            var jsonStart = responseText.IndexOf('[');
+            var jsonEnd = responseText.LastIndexOf(']');
+
+            if (jsonStart < 0 || jsonEnd < 0 || jsonEnd <= jsonStart)
+                return [];
+
+            var jsonText = responseText[jsonStart..(jsonEnd + 1)];
+
+            var items = JsonSerializer.Deserialize<List<ExtractedItem>>(jsonText, new JsonSerializerOptions
+            {
+                PropertyNameCaseInsensitive = true
+            });
+
+            if (items is null)
+                return [];
+
+            return items
+                .Where(i => !string.IsNullOrWhiteSpace(i.Title))
+                .Select(i => new RequirementDefinition
+                {
+                    Title = i.Title!.Trim(),
+                    Source = i.Source?.Trim(),
+                    Priority = NormalizePriority(i.Priority)
+                })
+                .ToList();
+        }
+        catch
+        {
+            return [];
+        }
+    }
+
+    private static string NormalizePriority(string? priority)
+    {
+        if (string.IsNullOrWhiteSpace(priority))
+            return "medium";
+
+        return priority.Trim().ToLowerInvariant() switch
+        {
+            "high" => "high",
+            "low" => "low",
+            _ => "medium"
+        };
+    }
+
+    private sealed class ExtractedItem
+    {
+        public string? Title { get; set; }
+        public string? Source { get; set; }
+        public string? Priority { get; set; }
+    }
+}

src/Spectra.CLI/Agent/Copilot/RequirementsExtractor.cs

@@ -29,26 +29,43 @@ public AnalyzeCommand() : base("analyze", "Analyze test coverage across document
             ["--auto-link"],
             "Scan automation code and update test automated_by fields");
 
+        var extractRequirementsOption = new Option<bool>(
+            ["--extract-requirements"],
+            "Extract testable requirements from documentation");
+
         AddOption(outputOption);
         AddOption(formatOption);
         AddOption(coverageOption);
         AddOption(autoLinkOption);
+        AddOption(extractRequirementsOption);
 
         this.SetHandler(async (context) =>
         {
             var output = context.ParseResult.GetValueForOption(outputOption);
             var format = context.ParseResult.GetValueForOption(formatOption);
             var coverage = context.ParseResult.GetValueForOption(coverageOption);
             var autoLink = context.ParseResult.GetValueForOption(autoLinkOption);
+            var extractReqs = context.ParseResult.GetValueForOption(extractRequirementsOption);
             var verbosity = context.ParseResult.GetValueForOption(GlobalOptions.VerbosityOption);
+            var dryRun = context.ParseResult.GetValueForOption(GlobalOptions.DryRunOption);
 
             var handler = new AnalyzeHandler(verbosity);
-            context.ExitCode = await handler.ExecuteAsync(
-                output,
-                format,
-                coverage,
-                autoLink,
-                context.GetCancellationToken());
+
+            if (extractReqs)
+            {
+                context.ExitCode = await handler.RunExtractRequirementsAsync(
+                    dryRun,
+                    context.GetCancellationToken());
+            }
+            else
+            {
+                context.ExitCode = await handler.ExecuteAsync(
+                    output,
+                    format,
+                    coverage,
+                    autoLink,
+                    context.GetCancellationToken());
+            }
         });
     }
 }

src/Spectra.CLI/Commands/Analyze/AnalyzeCommand.cs

@@ -202,6 +202,144 @@ await RunAutoLinkAsync(
         }
     }
 
+    /// <summary>
+    /// Extracts testable requirements from documentation using AI.
+    /// </summary>
+    public async Task<int> RunExtractRequirementsAsync(
+        bool dryRun,
+        CancellationToken ct = default)
+    {
+        try
+        {
+            var currentDir = Directory.GetCurrentDirectory();
+            var configPath = Path.Combine(currentDir, "spectra.config.json");
+
+            // Load config
+            SpectraConfig? config = null;
+            if (File.Exists(configPath))
+            {
+                var configJson = await File.ReadAllTextAsync(configPath, ct);
+                config = JsonSerializer.Deserialize<SpectraConfig>(configJson, new JsonSerializerOptions
+                {
+                    PropertyNameCaseInsensitive = true
+                });
+            }
+
+            if (config is null)
+            {
+                Console.Error.WriteLine("No spectra.config.json found. Run 'spectra init' first.");
+                return ExitCodes.Error;
+            }
+
+            // Auto-refresh document index
+            var indexService = new DocumentIndexService();
+            await indexService.EnsureIndexAsync(currentDir, config.Source, forceRebuild: false, ct);
+
+            // Load source documents
+            var docBuilder = new DocumentMapBuilder();
+            var documentMap = await docBuilder.BuildAsync(currentDir, ct);
+
+            if (documentMap.Documents.Count == 0)
+            {
+                Console.WriteLine("No documentation files found. Add docs to your source directory.");
+                return ExitCodes.Success;
+            }
+
+            // Load existing requirements
+            var reqsPath = Path.Combine(currentDir, config.Coverage.RequirementsFile);
+            var parser = new Spectra.Core.Parsing.RequirementsParser();
+            var existing = await parser.ParseAsync(reqsPath, ct);
+
+            // Get primary provider
+            var provider = config.Ai.Providers.FirstOrDefault(p => p.Enabled);
+            if (provider is null)
+            {
+                Console.Error.WriteLine("No AI provider configured. Run 'spectra init' to configure.");
+                return ExitCodes.Error;
+            }
+
+            // Extract requirements
+            if (_verbosity >= VerbosityLevel.Normal)
+            {
+                Console.WriteLine($"Extracting requirements from {documentMap.Documents.Count} document(s)...");
+            }
+
+            var extractor = new Agent.Copilot.RequirementsExtractor(
+                provider,
+                currentDir,
+                _verbosity >= VerbosityLevel.Normal ? Console.WriteLine : null);
+
+            var extracted = await extractor.ExtractAsync(documentMap.Documents, existing, ct);
+
+            if (extracted.Count == 0)
+            {
+                Console.WriteLine("No requirements found in documentation.");
+                return ExitCodes.Success;
+            }
+
+            // Merge and write
+            var writer = new Spectra.Core.Parsing.RequirementsWriter();
+            var result = writer.DetectDuplicates(existing, extracted);
+
+            if (dryRun)
+            {
+                Console.WriteLine();
+                Console.WriteLine($"Extracted {extracted.Count} requirement(s) (dry run — no files written):");
+                Console.WriteLine();
+
+                var withIds = writer.AllocateIds(existing, result.Merged);
+                foreach (var req in withIds)
+                {
+                    Console.WriteLine($"  {req.Id}  [{req.Priority}]  {req.Title}");
+                    Console.WriteLine($"          Source: {req.Source}");
+                }
+
+                if (result.Duplicates.Count > 0)
+                {
+                    Console.WriteLine();
+                    Console.WriteLine($"  {result.Duplicates.Count} duplicate(s) would be skipped");
+                }
+
+                return ExitCodes.Success;
+            }
+
+            var writeResult = await writer.MergeAndWriteAsync(reqsPath, extracted, ct);
+
+            // Report results
+            Console.WriteLine();
+            Console.WriteLine($"Requirements extraction complete:");
+            Console.WriteLine($"  New:        {writeResult.Merged.Count}");
+            Console.WriteLine($"  Duplicates: {writeResult.SkippedCount} (skipped)");
+            Console.WriteLine($"  Total:      {writeResult.TotalInFile}");
+            Console.WriteLine($"  File:       {Path.GetRelativePath(currentDir, reqsPath)}");
+
+            if (writeResult.Merged.Count > 0 && _verbosity >= VerbosityLevel.Normal)
+            {
+                Console.WriteLine();
+                foreach (var req in writeResult.Merged)
+                {
+                    Console.WriteLine($"  + {req.Id}  [{req.Priority}]  {req.Title}");
+                }
+            }
+
+            return ExitCodes.Success;
+        }
+        catch (OperationCanceledException)
+        {
+            Console.WriteLine("\nOperation cancelled.");
+            return ExitCodes.Cancelled;
+        }
+        catch (Exception ex)
+        {
+            Console.Error.WriteLine($"Error: {ex.Message}");
+            if (_verbosity >= VerbosityLevel.Detailed)
+            {
+                Console.Error.WriteLine(ex.StackTrace);
+            }
+            return ExitCodes.Error;
+        }
+    }
+
     /// <summary>
     /// Legacy doc-only coverage mode (when --coverage is not specified).
     /// </summary>