Merge branch '031-profile-format-file' - visible default profile format & customization guide

Author: angelovstanton

specs/031-profile-format-file/plan.md

@@ -0,0 +1,61 @@
+# Implementation Plan: 031 Profile Format File & Customization Guide
+
+## Architecture
+
+Two new embedded resources, one new loader class, and additions to three existing files.
+
+### New embedded resources
+
+```
+src/Spectra.CLI/Skills/Content/
+├── Profiles/_default.yaml        ← built-in default profile
+└── Docs/CUSTOMIZATION.md         ← customization guide
+```
+
+Both registered in `Spectra.CLI.csproj` via `<EmbeddedResource>`.
+
+### New code
+
+- `src/Spectra.CLI/Profile/ProfileFormatLoader.cs`
+  - Static helper `LoadFormat(string workingDirectory)` returns the JSON schema string used for `{{profile_format}}`.
+  - Resolution order:
+    1. `profiles/_default.yaml` on disk → parse YAML, extract `format` field.
+    2. Embedded resource `Spectra.CLI.Skills.Content.Profiles._default.yaml` → parse, extract `format`.
+  - On parse failure or missing `format`, log warning and fall back to embedded.
+  - Static helpers `LoadEmbeddedDefaultYaml()` and `LoadEmbeddedCustomizationGuide()` return raw text for use by init/update-skills.
+
+### Modified code
+
+- `src/Spectra.CLI/Agent/Copilot/GenerationAgent.cs`
+  - `BuildFullPrompt` accepts an optional `profileFormat` parameter (string). When non-null, it replaces the hardcoded `jsonExample`. The legacy fallback path and template-loader path both use it.
+  - `GenerateTestsAsync` calls `ProfileFormatLoader.LoadFormat(_basePath)` and passes the result.
+
+- `src/Spectra.CLI/Commands/Init/InitHandler.cs`
+  - New step `CreateDefaultProfileAsync` writes `profiles/_default.yaml`.
+  - New step `CreateCustomizationGuideAsync` writes `CUSTOMIZATION.md` at the project root.
+  - Both register their hashes in the skills manifest so `update-skills` can track them.
+
+- `src/Spectra.CLI/Commands/UpdateSkills/UpdateSkillsHandler.cs`
+  - Include `profiles/_default.yaml` and `CUSTOMIZATION.md` in the managed-file enumeration.
+
+- `src/Spectra.CLI/Spectra.CLI.csproj`
+  - Add `<EmbeddedResource Include="Skills\Content\Profiles\*.yaml" />` and `<EmbeddedResource Include="Skills\Content\Docs\*.md" />`.
+
+## Tests
+
+Added to `tests/Spectra.CLI.Tests/`:
+
+- `Profile/ProfileFormatLoaderTests.cs`
+  - `LoadFormat_ReturnsEmbeddedDefault_WhenNoFileExists`
+  - `LoadFormat_ReturnsFileContent_WhenDefaultFileExists`
+  - `LoadFormat_FallsBackToEmbedded_WhenFileIsMalformed`
+  - `LoadFormat_FallsBackToEmbedded_WhenFormatFieldMissing`
+- Extend `Commands/InitCommandTests.cs`
+  - `HandleAsync_CreatesDefaultProfile`
+  - `HandleAsync_CreatesCustomizationGuide`
+  - Verify both files appear in skills manifest.
+
+## Risks & decisions
+
+- The existing `ProfileConfig` in `Spectra.Core` is unrelated (it governs repository/suite markdown profiles). We deliberately do NOT add an `ai.profile` config field — the spec resolution narrows to `profiles/_default.yaml` on disk + embedded fallback. Named profiles via config are out of scope for this addendum and can be added later without breaking changes.
+- YAML parsing uses YamlDotNet (already a transitive dependency via `Spectra.Core`).

specs/031-profile-format-file/spec.md

@@ -0,0 +1,78 @@
+# Feature Specification: Default Profile Visibility & Customization Guide
+
+**Feature Branch**: `031-profile-format-file`
+**Created**: 2026-04-10
+**Status**: Draft
+**Depends on**: 030-prompt-templates, 004-test-generation-profile
+
+## Problem
+
+The `test-generation.md` prompt template references a `{{profile_format}}` placeholder that is silently filled by a hardcoded JSON schema baked into `GenerationAgent.BuildFullPrompt`. Users have no visibility into what the AI is asked to produce, no starting point to customize the output schema, and no single document explaining all the customization surfaces SPECTRA exposes.
+
+## User Scenarios & Testing
+
+### User Story 1 - Discoverable default format (Priority: P1)
+
+Test architects want to see — and edit — the JSON schema the AI uses for test generation, without grepping source code.
+
+**Why this priority**: This is the foundational visibility gap that drives the rest of the feature.
+
+**Independent Test**: After running `spectra init`, the file `profiles/_default.yaml` exists, contains a `format` field with the JSON schema, and includes inline documentation comments. Editing the `format` field changes what the AI receives on the next `spectra ai generate` run.
+
+**Acceptance Scenarios**:
+
+1. **Given** a fresh project, **When** the user runs `spectra init`, **Then** `profiles/_default.yaml` exists with a documented `format` field.
+2. **Given** a project with `profiles/_default.yaml` modified by the user, **When** generation runs, **Then** the AI prompt's `{{profile_format}}` placeholder resolves to the contents of the user's `format` field.
+3. **Given** `profiles/_default.yaml` is missing, **When** generation runs, **Then** the system uses the embedded built-in default and does not fail.
+
+### User Story 2 - One-stop customization guide (Priority: P1)
+
+Users want a single curated reference document explaining every place SPECTRA can be customized: profiles, prompt templates, behavior categories, config, branding, SKILLs, and agents.
+
+**Why this priority**: Customization surfaces already exist; without one map, they are effectively undiscoverable.
+
+**Independent Test**: After `spectra init`, `CUSTOMIZATION.md` exists at the project root and lists all customization surfaces with file paths and worked examples.
+
+**Acceptance Scenarios**:
+
+1. **Given** a fresh project, **When** the user runs `spectra init`, **Then** `CUSTOMIZATION.md` exists at the project root.
+2. **Given** an existing project where the user has not edited `CUSTOMIZATION.md`, **When** they run `spectra update-skills`, **Then** the file is refreshed to the latest version.
+3. **Given** an existing project where the user has edited `CUSTOMIZATION.md`, **When** they run `spectra update-skills`, **Then** the file is left untouched.
+
+### User Story 3 - Safe upgrade path (Priority: P2)
+
+Users want both new files tracked by the same hash-based update mechanism as SKILL files and prompt templates, so upgrades never silently overwrite their customizations.
+
+**Acceptance Scenarios**:
+
+1. **Given** `profiles/_default.yaml` matches the current built-in hash, **When** `spectra update-skills` runs, **Then** the file is updated to the latest built-in.
+2. **Given** the user has modified `profiles/_default.yaml`, **When** `spectra update-skills` runs, **Then** the file is preserved and reported as user-modified.
+
+### Edge Cases
+
+- `profiles/_default.yaml` exists but has malformed YAML → fall back to built-in embedded default and log a warning.
+- `profiles/_default.yaml` exists but has no `format` field → fall back to built-in embedded default.
+- Project does not have a `profiles/` directory at generation time → use built-in embedded default.
+
+## Requirements
+
+### Functional Requirements
+
+- **FR-001**: System MUST ship a built-in default profile YAML as an embedded resource that contains a `format` field whose value is the JSON schema sent to the AI as `{{profile_format}}`.
+- **FR-002**: `spectra init` MUST create `profiles/_default.yaml` from the embedded default unless the file already exists (without `--force`).
+- **FR-003**: When resolving `{{profile_format}}`, the system MUST check `profiles/_default.yaml` on disk before falling back to the embedded default.
+- **FR-004**: System MUST ship a built-in `CUSTOMIZATION.md` as an embedded resource covering all user-facing customization surfaces.
+- **FR-005**: `spectra init` MUST create `CUSTOMIZATION.md` at the project root from the embedded default unless the file already exists.
+- **FR-006**: Both `profiles/_default.yaml` and `CUSTOMIZATION.md` MUST be tracked in `.spectra/skills-manifest.json` so `spectra update-skills` can detect user modifications and preserve them.
+- **FR-007**: If `profiles/_default.yaml` is malformed or missing the `format` field, the system MUST gracefully fall back to the embedded built-in default and never crash test generation.
+
+### Key Entities
+
+- **ProfileFormatDocument**: A YAML document with a top-level `format` field (string containing the JSON schema sent to the AI) and an optional `fields` section documenting each output field for human readers.
+
+## Success Criteria
+
+- **SC-001**: A user can change the JSON schema sent to the AI for test generation by editing exactly one file (`profiles/_default.yaml`) without writing any code.
+- **SC-002**: A new user can locate every customization surface in SPECTRA from a single document at the project root in under 60 seconds.
+- **SC-003**: 100% of `dotnet test` runs (including new tests added by this feature) pass after implementation.
+- **SC-004**: `spectra update-skills` never overwrites a user-modified `profiles/_default.yaml` or `CUSTOMIZATION.md`.

specs/031-profile-format-file/tasks.md

@@ -0,0 +1,32 @@
+# Tasks: 031 Profile Format File & Customization Guide
+
+## T1 — Add embedded default profile YAML
+- Create `src/Spectra.CLI/Skills/Content/Profiles/_default.yaml` with documented `format` field + `fields` reference section.
+- Update `Spectra.CLI.csproj` to include `Skills\Content\Profiles\*.yaml` as embedded resource.
+
+## T2 — Add embedded customization guide
+- Create `src/Spectra.CLI/Skills/Content/Docs/CUSTOMIZATION.md`.
+- Update `Spectra.CLI.csproj` to include `Skills\Content\Docs\*.md` as embedded resource.
+
+## T3 — Implement ProfileFormatLoader
+- New file `src/Spectra.CLI/Profile/ProfileFormatLoader.cs`.
+- Methods: `LoadFormat(string workingDirectory)`, `LoadEmbeddedDefaultYaml()`, `LoadEmbeddedCustomizationGuide()`.
+- 3-tier fallback with malformed-file safety.
+
+## T4 — Wire ProfileFormatLoader into GenerationAgent
+- Update `BuildFullPrompt` to accept optional `profileFormat` string and use it in both code paths.
+- Update `GenerateTestsAsync` to load and pass it.
+
+## T5 — Init handler creates new files
+- Add `CreateDefaultProfileAsync` and `CreateCustomizationGuideAsync` to `InitHandler`.
+- Both register hashes in `.spectra/skills-manifest.json`.
+
+## T6 — update-skills tracks new files
+- Update `UpdateSkillsHandler` to include both new files in its managed list.
+
+## T7 — Tests
+- Add `tests/Spectra.CLI.Tests/Profile/ProfileFormatLoaderTests.cs` (4 tests).
+- Add init tests for new files in `Commands/InitCommandTests.cs` (2 tests).
+
+## T8 — Verify
+- Run `dotnet test` and confirm all tests pass.

src/Spectra.CLI/Agent/Copilot/GenerationAgent.cs

@@ -4,6 +4,7 @@
 using GitHub.Copilot.SDK;
 using Microsoft.Extensions.AI;
 using Spectra.CLI.Agent.Tools;
+using Spectra.CLI.Profile;
 using Spectra.CLI.Prompts;
 using Spectra.Core.Index;
 using Spectra.Core.Models;
@@ -133,8 +134,11 @@ public async Task<GenerationResult> GenerateTestsAsync(
                 }
             });
 
-            // Build the combined prompt with system instructions and user request
-            var fullPrompt = BuildFullPrompt(prompt, requestedCount, criteriaContext);
+            // Build the combined prompt with system instructions and user request.
+            // The profile format (JSON schema sent to the AI) is resolved from
+            // profiles/_default.yaml on disk if present, else the embedded default.
+            var profileFormat = ProfileFormatLoader.LoadFormat(_basePath);
+            var fullPrompt = BuildFullPrompt(prompt, requestedCount, criteriaContext, profileFormat: profileFormat);
 
             // Send and wait for the complete response
             _onStatus?.Invoke("Starting AI generation...");
@@ -246,27 +250,14 @@ private void SaveDebugResponse(string responseText)
     }
 
     internal static string BuildFullPrompt(string userPrompt, int requestedCount, string? criteriaContext = null,
-        PromptTemplateLoader? templateLoader = null)
+        PromptTemplateLoader? templateLoader = null, string? profileFormat = null)
     {
-        var jsonExample = """
-            [
-              {
-                "id": "TC-XXX",
-                "title": "Descriptive title based on documentation",
-                "priority": "high|medium|low",
-                "tags": ["tag1", "tag2"],
-                "component": "component-name",
-                "preconditions": "Setup requirements",
-                "steps": ["Step 1", "Step 2", "Step 3"],
-                "expected_result": "Specific outcome from documentation",
-                "test_data": "Test data if needed",
-                "source_refs": ["docs/file.md#Section-Name"],
-                "scenario_from_doc": "Quote or paraphrase the documented behavior",
-                "estimated_duration": "5m",
-                "criteria": ["AC-ID-1", "AC-ID-2"]
-              }
-            ]
-            """;
+        // The JSON output schema sent to the AI. Prefer the caller-supplied
+        // profileFormat (resolved from profiles/_default.yaml on disk or the
+        // embedded default by ProfileFormatLoader). When null (legacy callers
+        // and unit tests), fall back to the embedded default to keep behavior
+        // identical.
+        var jsonExample = profileFormat ?? ProfileFormatLoader.LoadFormat(Directory.GetCurrentDirectory());
 
         if (templateLoader is not null)
         {

src/Spectra.CLI/Commands/Init/InitHandler.cs

@@ -3,6 +3,7 @@
 using System.Text.Json;
 using Spectra.CLI.Infrastructure;
 using Spectra.CLI.Output;
+using Spectra.CLI.Profile;
 using Spectra.CLI.Source;
 using Spectra.Core.Config;
 using Spectra.Core.Models.Config;
@@ -79,6 +80,9 @@ public async Task<int> HandleAsync(bool force, bool skipSkills = false, Cancella
             // Create prompt templates
             await CreatePromptTemplatesAsync(force, ct);
 
+            // Create default profile and customization guide
+            await CreateDefaultProfileAndCustomizationAsync(force, ct);
+
             // Create dashboard deployment workflow
             await CreateDeployWorkflowAsync(ct);
 
@@ -589,6 +593,48 @@ private async Task CreatePromptTemplatesAsync(bool force, CancellationToken ct)
         await manifestStore.SaveAsync(manifest, ct);
     }
 
+    private async Task CreateDefaultProfileAndCustomizationAsync(bool force, CancellationToken ct)
+    {
+        var manifestStore = new Skills.SkillsManifestStore(_workingDirectory);
+        var manifest = await manifestStore.LoadAsync(ct);
+
+        // profiles/_default.yaml
+        var profilesDir = Path.Combine(_workingDirectory, "profiles");
+        Directory.CreateDirectory(profilesDir);
+        var profilePath = Path.Combine(profilesDir, "_default.yaml");
+        var profileContent = ProfileFormatLoader.LoadEmbeddedDefaultYaml();
+        var profileRelative = Path.Combine("profiles", "_default.yaml");
+
+        if (!File.Exists(profilePath) || force)
+        {
+            await File.WriteAllTextAsync(profilePath, profileContent, ct);
+            _logger.LogInformation("Created default profile: {Path}", profileRelative);
+        }
+        else
+        {
+            _logger.LogDebug("Default profile already exists, skipping: {Path}", profilePath);
+        }
+        manifest.Files[profileRelative] = Infrastructure.FileHasher.ComputeHash(profileContent);
+
+        // CUSTOMIZATION.md (project root)
+        var customizationPath = Path.Combine(_workingDirectory, "CUSTOMIZATION.md");
+        var customizationContent = ProfileFormatLoader.LoadEmbeddedCustomizationGuide();
+        const string customizationRelative = "CUSTOMIZATION.md";
+
+        if (!File.Exists(customizationPath) || force)
+        {
+            await File.WriteAllTextAsync(customizationPath, customizationContent, ct);
+            _logger.LogInformation("Created customization guide: {Path}", customizationRelative);
+        }
+        else
+        {
+            _logger.LogDebug("Customization guide already exists, skipping: {Path}", customizationPath);
+        }
+        manifest.Files[customizationRelative] = Infrastructure.FileHasher.ComputeHash(customizationContent);
+
+        await manifestStore.SaveAsync(manifest, ct);
+    }
+
     private async Task CreateVsCodeMcpConfigAsync(CancellationToken ct)
     {
         var mcpConfigPath = Path.Combine(_workingDirectory, VsCodeMcpPath);

src/Spectra.CLI/Commands/UpdateSkills/UpdateSkillsHandler.cs

@@ -1,5 +1,6 @@
 using Spectra.CLI.Infrastructure;
 using Spectra.CLI.Output;
+using Spectra.CLI.Profile;
 using Spectra.CLI.Prompts;
 using Spectra.CLI.Skills;
 
@@ -62,6 +63,20 @@ public async Task<int> ExecuteAsync(CancellationToken ct = default)
             await ProcessFileAsync(templatePath, content, manifest, updated, unchanged, skipped, ct);
         }
 
+        // Process default profile (profiles/_default.yaml)
+        var defaultProfilePath = Path.Combine(currentDir, "profiles", "_default.yaml");
+        await ProcessFileAsync(
+            defaultProfilePath,
+            ProfileFormatLoader.LoadEmbeddedDefaultYaml(),
+            manifest, updated, unchanged, skipped, ct);
+
+        // Process customization guide (CUSTOMIZATION.md at project root)
+        var customizationPath = Path.Combine(currentDir, "CUSTOMIZATION.md");
+        await ProcessFileAsync(
+            customizationPath,
+            ProfileFormatLoader.LoadEmbeddedCustomizationGuide(),
+            manifest, updated, unchanged, skipped, ct);
+
         // Save updated manifest
         await manifestStore.SaveAsync(manifest, ct);