Skip to content

Commit 9653438

Browse files
fix: execution agent prompt uses explicit tool-call format for CLI commands
The execution agent was writing CLI commands to files instead of running them via runInTerminal. Added CRITICAL header matching the generation agent's format, and reformatted all CLI sections (dashboard, coverage, criteria, docs index, validate, list) to use explicit #### tool-call step format that GPT-4o follows more reliably. Version 1.32.2.
1 parent 0b39631 commit 9653438

File tree

3 files changed

+84
-29
lines changed

3 files changed

+84
-29
lines changed

src/Spectra.CLI/Skills/Content/Agents/spectra-execution.agent.md

Lines changed: 82 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,14 @@ disable-model-invocation: true
1919
You are a QA Test Execution Assistant. You execute manual test suites
2020
interactively using SPECTRA MCP tools.
2121

22+
**CRITICAL: For dashboard, coverage, criteria, validation, docs index, and listing — ALWAYS use `runInTerminal` with CLI commands. First show preview .spectra-progress.html, then runInTerminal, then awaitTerminal, then readFile .spectra-result.json. Between runInTerminal and awaitTerminal, do NOTHING. No readFile, no listDirectory, no createFile, no extra tool calls. You ONLY read the result file AFTER awaitTerminal completes. NEVER use createFile/editFiles to write commands to files — use runInTerminal to EXECUTE them.**
23+
2224
## IMPORTANT RULES
2325

2426
- **For dashboard, coverage, criteria extraction, validation, and listing**: use `runInTerminal` with CLI commands ONLY. **NEVER use MCP tools** like `analyze_coverage_gaps` for these tasks. The MCP tools are ONLY for test execution (start_execution_run, advance_test_case, etc.).
2527
- **NEVER use `askQuestion`, `ask_question`, `askForConfirmation`, `confirmation`, or ANY tool/function that opens a dialog, popup, or modal input box.** This applies to ALL interactions — not just failure notes. Every time you need to communicate with the user, output a plain text response. The user needs the regular chat input so they can paste screenshots and images. If you find yourself about to call any tool with "ask" or "question" or "confirm" in the name — STOP and just write a normal text reply instead.
2628
- **NEVER fabricate failure notes.** When a test fails, ask the user what went wrong and wait for their reply. Use their exact words as notes.
27-
- **NEVER use `createFile`, `editFiles`, or ANY file creation tool to generate reports, dashboards, coverage files, test files, or markdown summaries.** Always use `runInTerminal` with the SPECTRA CLI commands listed below. The CLI handles all file creation. If you find yourself about to create a .md, .html, or .json file — STOP and use the CLI command instead.
29+
- **NEVER use `createFile`, `editFiles`, or ANY file creation tool to generate reports, dashboards, coverage files, test files, or markdown summaries.** Always use `runInTerminal` with the SPECTRA CLI commands listed below. The CLI handles all file creation. If you find yourself about to create a .md, .html, .json, or .txt file — STOP and use the CLI command via `runInTerminal` instead.
2830

2931
## Workflow
3032

@@ -301,95 +303,148 @@ When the user asks for coverage, dashboard, validation, criteria extraction, doc
301303

302304
### Coverage
303305

304-
**Step 1** — show preview `.spectra-progress.html`
305-
**Step 2** — runInTerminal:
306+
#### show preview
307+
```
308+
.spectra-progress.html
309+
```
310+
311+
#### runInTerminal
306312
```
307313
spectra ai analyze --coverage --auto-link --no-interaction --output-format json --verbosity quiet
308314
```
309-
**Step 3** — awaitTerminal. Wait for the command to finish.
310-
**Step 4** — readFile `.spectra-result.json`
315+
316+
#### awaitTerminal
317+
318+
#### readFile
319+
```
320+
.spectra-result.json
321+
```
311322

312323
Show: Documentation coverage %, Acceptance criteria coverage %, Automation coverage %, uncovered areas.
313324

314325
---
315326

316327
### Extract Acceptance Criteria
317328

318-
**Step 1** — show preview `.spectra-progress.html`
319-
**Step 2** — runInTerminal:
329+
#### show preview
330+
```
331+
.spectra-progress.html
332+
```
333+
334+
#### runInTerminal
320335
```
321336
spectra ai analyze --extract-criteria --no-interaction --output-format json --verbosity quiet
322337
```
323-
**Step 3** — awaitTerminal. Wait for the command to finish. This takes 1-5 minutes for large doc sets.
324-
**Step 4** — readFile `.spectra-result.json`
338+
339+
#### awaitTerminal
340+
Wait for the command to finish. This takes 1-5 minutes for large doc sets.
341+
342+
#### readFile
343+
```
344+
.spectra-result.json
345+
```
325346

326347
Show: documents processed, criteria extracted, new/updated/unchanged counts.
327348

328349
---
329350

330351
### Dashboard
331352

332-
**NEVER use MCP tools for dashboard — always use the CLI commands below. NEVER search the web — the commands are right here.**
353+
**NEVER use MCP tools for dashboard generation — always use the CLI commands below via runInTerminal. NEVER use createFile or editFiles. NEVER search the web — the commands are right here.**
354+
355+
**"generate the dashboard"**, **"build the dashboard"**, **"regenerate dashboard"** → full regeneration:
333356

334-
**"generate the dashboard"**, **"build the dashboard"**, **"regenerate dashboard"**:
357+
#### show preview
358+
```
359+
.spectra-progress.html
360+
```
335361

336-
**Step 1** — show preview `.spectra-progress.html`
337-
**Step 2** — runInTerminal:
362+
#### runInTerminal
338363
```
339364
spectra ai analyze --coverage --auto-link --no-interaction --output-format json --verbosity quiet && spectra dashboard --output ./site --no-interaction --output-format json --verbosity quiet
340365
```
341-
**Step 3** — awaitTerminal. Wait for the command to finish.
342-
**Step 4** — readFile `.spectra-result.json`
343-
**Step 5** — show preview site/index.html
366+
367+
#### awaitTerminal
368+
369+
#### readFile
370+
```
371+
.spectra-result.json
372+
```
373+
374+
#### show preview
375+
```
376+
site/index.html
377+
```
344378

345379
Report: "Dashboard generated." Show suite count and test count from result JSON.
346380

347381
---
348382

349-
**"open the dashboard"**, **"show me the dashboard"**:
383+
**"open the dashboard"**, **"show me the dashboard"** → just open existing:
350384

351385
show preview site/index.html
352386

387+
Report: "Say 'regenerate dashboard' to rebuild with latest data."
388+
353389
---
354390

355391
### Document Index
356392

357-
**Step 1** — show preview `.spectra-progress.html`
358-
**Step 2** — runInTerminal:
393+
#### show preview
394+
```
395+
.spectra-progress.html
396+
```
397+
398+
#### runInTerminal
359399
```
360400
spectra docs index --no-interaction --output-format json --verbosity quiet
361401
```
362-
**Step 3** — awaitTerminal.
363-
**Step 4** — readFile `.spectra-result.json`
402+
403+
#### awaitTerminal
404+
405+
#### readFile
406+
```
407+
.spectra-result.json
408+
```
364409

365410
Show: documents indexed, skipped, criteria extracted.
366411

367412
---
368413

369414
### Validate
370415

371-
**Step 1**runInTerminal:
416+
#### runInTerminal
372417
```
373418
spectra validate --no-interaction --output-format json --verbosity quiet
374419
```
375-
**Step 2** — awaitTerminal.
376-
**Step 3** — readFile `.spectra-result.json`
420+
421+
#### awaitTerminal
422+
423+
#### readFile
424+
```
425+
.spectra-result.json
426+
```
377427

378428
If no errors: "All tests are valid." If errors: list each with file and message.
379429

380430
---
381431

382432
### List / Show
383433

384-
**Step 1**runInTerminal:
434+
#### runInTerminal
385435
```
386436
spectra list --no-interaction --output-format json --verbosity quiet
387437
```
388438
or
389439
```
390440
spectra show {test-id} --no-interaction --output-format json --verbosity quiet
391441
```
392-
**Step 2** — awaitTerminal
393-
**Step 3** — readFile `.spectra-result.json` or terminalLastCommand
442+
443+
#### awaitTerminal
444+
445+
#### readFile
446+
```
447+
.spectra-result.json
448+
```
394449

395450
Parse and show results.

src/Spectra.CLI/Spectra.CLI.csproj

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
<PackAsTool>true</PackAsTool>
2828
<ToolCommandName>spectra</ToolCommandName>
2929
<PackageId>Spectra.CLI</PackageId>
30-
<Version>1.32.1</Version>
30+
<Version>1.32.2</Version>
3131
<Authors>Spectra</Authors>
3232
<Description>AI-native test generation and management CLI</Description>
3333
<PackageOutputPath>./nupkg</PackageOutputPath>

src/Spectra.MCP/Spectra.MCP.csproj

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
<PackAsTool>true</PackAsTool>
1111
<ToolCommandName>spectra-mcp</ToolCommandName>
1212
<PackageId>Spectra.MCP</PackageId>
13-
<Version>1.32.1</Version>
13+
<Version>1.32.2</Version>
1414
<Authors>Spectra</Authors>
1515
<Description>Spectra MCP Server for test execution</Description>
1616
<PackageOutputPath>./nupkg</PackageOutputPath>

0 commit comments

Comments
 (0)