Skip to content

Commit 8c1ae9e

Browse files
feat(v1.47.0): live progress bars for generate + update (spec 041)
Long generate runs (--count 40+) and update runs now show real per-batch generation and per-test critic progress in two places: - Terminal: Spectre.Console two-task progress bar via the new ProgressReporter.ProgressTwoTaskAsync wrapper. Generation handle advances per batch; verification handle advances per critic call with the latest test ID + verdict in the description. Update handler shows a single per-proposal bar. - Browser progress page: a new progress section in .spectra-progress.html driven by an in-flight `progress` object inside .spectra-result.json. Verification bar is dimmed during the generation phase, then activated. Picked up by the existing 1.5s auto-refresh; no new I/O during generation phase (reuses the existing per-batch result file flush). Suppression rules: progress bars are suppressed under --output-format json, --verbosity quiet, AND non-interactive stdout (no TTY / piped). SKILL/CI invocations stay ANSI-free. ProgressManager.Complete()/Fail() now clear the in-flight ProgressSnapshot before the final write, so the final result file always has runSummary instead of stale progress data. +17 unit tests covering suppression rules, snapshot clear-on-complete, and HTML rendering for generating/verifying/updating phases. Out of scope: ETA, per-test progress within a batch, formal cancel.
1 parent a088a03 commit 8c1ae9e

File tree

21 files changed

+1725
-12
lines changed

21 files changed

+1725
-12
lines changed

CHANGELOG.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [1.47.0] - 2026-04-11
9+
10+
### Added
11+
- **Live progress bars for `spectra ai generate` and `spectra ai update`** (spec 041). Long runs (`--count 40+`) now show real progress feedback in two places:
12+
- **Terminal**: a `Generating tests` Spectre.Console progress bar advances per batch, followed by a `Verifying tests` bar that increments once per critic call (showing the most recent test ID and verdict). For `update`, an `Updating tests` bar advances per applied proposal.
13+
- **Browser progress page** (`.spectra-progress.html`): a new live progress section renders the same data driven by an in-flight `progress` object inside `.spectra-result.json`, picked up on the existing 1.5s auto-refresh cycle. Verification bar is dimmed during the generation phase, then activated.
14+
- **`progress` field in `.spectra-result.json`** — present only while a run is in flight (snapshot of phase, target, generated, verified, current batch, total batches, last test ID, last verdict). Removed automatically by `ProgressManager.Complete()` / `Fail()` so the final result file shows `runSummary` instead. Schema documented in `specs/041-progress-bars/contracts/result-json-progress-schema.md`.
15+
- **`ProgressReporter.ProgressTwoTaskAsync(...)`** — new wrapper that creates two sequential Spectre tasks inside a single live region. Handles passed to the action implement a small `IProgressTaskHandle` abstraction so command handlers don't take a direct dependency on Spectre internals.
16+
- **`ProgressSnapshot` + `ProgressPhase` model** at `src/Spectra.CLI/Progress/ProgressSnapshot.cs`.
17+
- **+17 unit tests**`ProgressBarTests`, `ProgressManagerProgressFieldTests`, `ProgressPageProgressBarTests` covering suppression rules, progress field clear-on-complete/fail, and HTML rendering for generation/verifying/updating phases.
18+
19+
### Changed
20+
- **`ProgressReporter` suppression rules** — now suppresses progress bars not just under `--output-format json`, but also under `--verbosity quiet` and when stdout is non-interactive (redirected, piped, or no TTY). SKILL/CI invocations remain ANSI-free.
21+
- **`ProgressReporter.StatusAsync`** — short-circuits when the reporter is inside an active `ProgressTwoTaskAsync` block, so existing inline `_progress.StatusAsync(...)` spinners inside the batch loop don't conflict with the outer progress bars.
22+
- **`UpdateHandler.ApplyChangesAsync`** — gained an optional `onProposalApplied` callback parameter for per-proposal progress reporting. Existing call sites unaffected.
23+
24+
### Notes
25+
- ETA / time-remaining display, per-test progress within a single generation batch, and formal cancel-with-cleanup on Ctrl+C are explicitly out of scope. See `specs/041-progress-bars/spec.md`.
26+
- `--output-format json` and `--verbosity quiet` behavior is unchanged: SKILLs and CI scripts see no progress-bar output on stdout.
27+
828
## [1.46.1] - 2026-04-11
929

1030
### Fixed

docs/cli-reference.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,15 @@ spectra ai generate checkout --no-interaction --output-format json # CI pipel
188188

189189
Session state is stored in `.spectra/session.json` and expires after 1 hour.
190190

191+
**Live progress bars** (spec 041) — at default verbosity, long runs render two
192+
sequential Spectre.Console progress bars in the terminal: a `Generating tests`
193+
bar that advances per batch, then a `Verifying tests` bar that advances per
194+
critic call (showing the most recent test ID and verdict). The same data is
195+
mirrored to `.spectra-progress.html` via an in-flight `progress` object inside
196+
`.spectra-result.json`. Progress bars are automatically suppressed under
197+
`--output-format json`, `--verbosity quiet`, or non-interactive stdout (piped
198+
or redirected) so SKILL/CI output remains clean.
199+
191200
User-described tests are marked with `grounding.verdict: manual` and `source: user-described`.
192201

193202
When a project has documentation in `docs/` and acceptance criteria in `docs/criteria/`, `--from-description` runs in **doc-aware mode**: it best-effort loads matching docs (capped at 3 docs × 8000 chars) and matching `.criteria.yaml` entries as formatting context, then populates the new test's `source_refs` (with the doc paths used) and `criteria` fields (with any IDs the AI matches to your description). The grounding verdict stays `manual` — doc context is used for terminology and navigation alignment only, never for verification. If no docs or criteria exist, the flow is identical to the no-context behavior.
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Specification Quality Checklist: Generation & Verification Progress Bars
2+
3+
**Purpose**: Validate specification completeness and quality before proceeding to planning
4+
**Created**: 2026-04-11
5+
**Feature**: [spec.md](../spec.md)
6+
7+
## Content Quality
8+
9+
- [x] No implementation details (languages, frameworks, APIs)
10+
- [x] Focused on user value and business needs
11+
- [x] Written for non-technical stakeholders
12+
- [x] All mandatory sections completed
13+
14+
## Requirement Completeness
15+
16+
- [x] No [NEEDS CLARIFICATION] markers remain
17+
- [x] Requirements are testable and unambiguous
18+
- [x] Success criteria are measurable
19+
- [x] Success criteria are technology-agnostic (no implementation details)
20+
- [x] All acceptance scenarios are defined
21+
- [x] Edge cases are identified
22+
- [x] Scope is clearly bounded
23+
- [x] Dependencies and assumptions identified
24+
25+
## Feature Readiness
26+
27+
- [x] All functional requirements have clear acceptance criteria
28+
- [x] User scenarios cover primary flows
29+
- [x] Feature meets measurable outcomes defined in Success Criteria
30+
- [x] No implementation details leak into specification
31+
32+
## Notes
33+
34+
- Spec mentions Spectre.Console and `.spectra-progress.html` / `.spectra-result.json` filenames, but these are existing project surfaces (not new technology choices) and are documented in the Assumptions section as such — so they are treated as scoped context, not implementation leakage.
35+
- All items pass on first iteration. Ready for `/speckit.plan`.
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Contract: `progress` field in `.spectra-result.json`
2+
3+
**Producer**: `Spectra.CLI` `GenerateHandler`, `UpdateHandler` via `ProgressManager.Update(result)`.
4+
**Consumers**: `ProgressPageWriter` (writes `.spectra-progress.html`); SKILL/CI scripts that watch the result file.
5+
6+
## Field
7+
8+
`progress` is OPTIONAL and present only while a run is in flight. It MUST be absent from the final result file (after `Complete()` or `Fail()`).
9+
10+
## Schema
11+
12+
```json
13+
{
14+
"progress": {
15+
"phase": "generating | verifying | updating", // required
16+
"testsTarget": 40, // required, integer ≥ 0
17+
"testsGenerated": 24, // required, integer ≥ 0, ≤ testsTarget
18+
"testsVerified": 0, // required, integer ≥ 0, ≤ testsGenerated
19+
"currentBatch": 3, // required, integer ≥ 1, ≤ totalBatches
20+
"totalBatches": 5, // required, integer ≥ 1
21+
"lastTestId": "TC-124", // optional, string or absent
22+
"lastVerdict": "grounded" // optional; one of grounded|partial|hallucinated; absent unless phase = verifying
23+
}
24+
}
25+
```
26+
27+
## Stability Guarantees
28+
29+
- Field names are camelCase (matches existing `JsonResultWriter` policy).
30+
- Consumers MUST tolerate the entire `progress` object being absent.
31+
- Consumers MUST tolerate `lastTestId` / `lastVerdict` being absent.
32+
- Adding new optional fields to `progress` in future revisions is non-breaking.
33+
- Removing or renaming fields is breaking and requires a spec increment.
34+
35+
## Refresh Cadence
36+
37+
- Generation phase: written after each batch completes (existing per-batch write site, no extra I/O).
38+
- Verification phase: written after each individual critic call completes (per-test write — acceptable since critic calls take 4–6s).
39+
- Update phase: written after each proposal application.
40+
- Final write (with `progress` absent): one write at end of run.
41+
42+
## Reading Safety
43+
44+
`ProgressPageWriter` writes `.spectra-result.json` via temp file + atomic move (existing path in `ProgressManager.FlushWriteFile`). Readers performing whole-file reads will always see a consistent snapshot — either the previous version or the new one, never a half-written file.
Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# Phase 1 Data Model: Progress Bars
2+
3+
## ProgressSnapshot (record)
4+
5+
Lives in `src/Spectra.CLI/Progress/ProgressSnapshot.cs` (NEW). Attached to `GenerateResult` and `UpdateResult` via an optional `Progress` field. Serialized as `progress` in `.spectra-result.json`.
6+
7+
### Fields
8+
9+
| Field | Type | Required | Notes |
10+
|---|---|---|---|
11+
| `Phase` | enum `ProgressPhase` | yes | `Generating` \| `Verifying` \| `Updating` |
12+
| `TestsTarget` | int | yes | Total tests this run intends to produce/process |
13+
| `TestsGenerated` | int | yes | Cumulative count after each completed batch (0 in `Updating` phase) |
14+
| `TestsVerified` | int | yes | Cumulative count after each completed critic call (0 outside `Verifying` phase) |
15+
| `CurrentBatch` | int | yes | 1-based index of the in-progress (or just-completed) unit. For `Generating`: batch index. For `Verifying`: stays at the final batch index. For `Updating`: proposal index. |
16+
| `TotalBatches` | int | yes | Total units. For `Generating`: `ceil(TestsTarget / GenerationBatchSize)`. For `Updating`: total proposal count. |
17+
| `LastTestId` | string? | no | Most recent test handled (test ID, e.g. `TC-118`) |
18+
| `LastVerdict` | string? | no | Critic verdict for `LastTestId`: `grounded` \| `partial` \| `hallucinated`. Null outside `Verifying`. |
19+
20+
### Lifecycle / State Transitions
21+
22+
```
23+
null
24+
25+
│ handler creates snapshot at start of generation
26+
27+
Phase=Generating, TestsGenerated=0, TestsVerified=0, CurrentBatch=1, TotalBatches=N
28+
29+
│ after each batch completes: TestsGenerated += batch.RequestedCount, CurrentBatch++
30+
31+
Phase=Generating, TestsGenerated=TestsTarget, CurrentBatch=N (terminal generation state)
32+
33+
│ critic phase begins (only if not --skip-critic)
34+
35+
Phase=Verifying, TestsVerified=0
36+
37+
│ after each critic call: TestsVerified++, LastTestId, LastVerdict updated
38+
39+
Phase=Verifying, TestsVerified=TestsTarget
40+
41+
│ ProgressManager.Complete() or .Fail()
42+
43+
null (cleared from final result file)
44+
```
45+
46+
For `UpdateHandler`: snapshot starts as `Phase=Updating`, advances per proposal apply, cleared on completion.
47+
48+
### Validation Rules
49+
50+
- `TestsGenerated <= TestsTarget`
51+
- `TestsVerified <= TestsGenerated` (you cannot verify what wasn't generated)
52+
- `CurrentBatch <= TotalBatches`
53+
- `LastVerdict ∈ {grounded, partial, hallucinated, null}`
54+
- `Phase``LastVerdict` consistency: `LastVerdict` MUST be null when `Phase != Verifying`
55+
56+
These are invariants of the producing handler — not validated at deserialization (the consumer is the progress page, which is tolerant of missing fields).
57+
58+
## Glossary Note
59+
60+
**"Chunks" vs "Proposals"**: The original spec text referenced update "chunks". The actual `UpdateHandler` has no chunk concept — it does one classification batch and then a per-proposal apply loop. The Update progress bar tracks proposals; `CurrentBatch` / `TotalBatches` carry proposal counts in `Phase=Updating`. FR-005 is satisfied by per-proposal advancement.
61+
62+
## Result File Shape (full example)
63+
64+
In-flight (mid-generation):
65+
66+
```json
67+
{
68+
"status": "generating",
69+
"command": "generate",
70+
"suite": "checkout",
71+
"progress": {
72+
"phase": "generating",
73+
"testsTarget": 40,
74+
"testsGenerated": 24,
75+
"testsVerified": 0,
76+
"currentBatch": 3,
77+
"totalBatches": 5,
78+
"lastTestId": "TC-124",
79+
"lastVerdict": null
80+
}
81+
}
82+
```
83+
84+
In-flight (mid-verification):
85+
86+
```json
87+
{
88+
"status": "generating",
89+
"command": "generate",
90+
"suite": "checkout",
91+
"progress": {
92+
"phase": "verifying",
93+
"testsTarget": 40,
94+
"testsGenerated": 40,
95+
"testsVerified": 18,
96+
"currentBatch": 5,
97+
"totalBatches": 5,
98+
"lastTestId": "TC-118",
99+
"lastVerdict": "grounded"
100+
}
101+
}
102+
```
103+
104+
Final (after `Complete()`): `progress` field is omitted (`JsonIgnoreCondition.WhenWritingNull`); `runSummary` is present instead.

0 commit comments

Comments
 (0)