Commit 60e6348
committed
fix(generate): configurable per-batch timeout + batch size + .spectra-debug.log
The 5-minute SDK timeout in CopilotGenerationAgent has been hardcoded since
spec 009. Fast generators (gpt-4o, gpt-4o-mini) fit 30 tests per batch in
under 5 minutes; slower / reasoning models or large Azure deployments do
not. Symptom: --count 100 splits into 4 batches of 30, the first batch hits
the 5-minute ceiling, generation fails. --count 1 works because one test
fits in any budget.
AiConfig (new optional fields):
- generation_timeout_minutes (default 5) — per-batch SDK call timeout
- generation_batch_size (default 30) — number of tests per AI call
- debug_log_enabled (default true) — gates the per-batch diagnostic file
GenerationAgent.GenerateTestsAsync:
- Reads timeout from config; minimum 1 minute
- Logs each batch START / OK / TIMEOUT to .spectra-debug.log with the
configured timeout, model name, provider, requested count, and elapsed
seconds (best-effort, never throws)
- Surfaces the configured timeout and batch size in the live status message
- Timeout error message now lists the actual configured minutes, the model
name, the batch size, and three concrete remediation options with
copy-paste-ready spectra.config.json snippets
GenerateHandler:
- Replaces hardcoded BatchSize const with DefaultBatchSize fallback
- Reads ai.generation_batch_size from config when > 0
- Renders "Batch N/M" in progress messages (was "Batch N")
- Loop math unchanged
For users hitting the timeout with a slow model, the workaround is now in
config — no code change required:
"ai": {
"generation_timeout_minutes": 15,
"generation_batch_size": 10
}
The .spectra-debug.log file shows exactly which batch was slow and why.1 parent 0da74c6 commit 60e6348
File tree
3 files changed
+80
-8
lines changed- src
- Spectra.CLI
- Agent/Copilot
- Commands/Generate
- Spectra.Core/Models/Config
3 files changed
+80
-8
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
170 | 170 | | |
171 | 171 | | |
172 | 172 | | |
173 | | - | |
174 | | - | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
175 | 181 | | |
176 | 182 | | |
177 | | - | |
| 183 | + | |
178 | 184 | | |
179 | 185 | | |
180 | 186 | | |
181 | 187 | | |
| 188 | + | |
| 189 | + | |
182 | 190 | | |
183 | 191 | | |
184 | 192 | | |
| |||
219 | 227 | | |
220 | 228 | | |
221 | 229 | | |
| 230 | + | |
| 231 | + | |
222 | 232 | | |
223 | 233 | | |
224 | 234 | | |
225 | | - | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
226 | 245 | | |
227 | 246 | | |
228 | 247 | | |
| |||
269 | 288 | | |
270 | 289 | | |
271 | 290 | | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
272 | 311 | | |
273 | 312 | | |
274 | 313 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | | - | |
| 35 | + | |
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
| |||
522 | 522 | | |
523 | 523 | | |
524 | 524 | | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
525 | 533 | | |
526 | 534 | | |
527 | 535 | | |
528 | | - | |
| 536 | + | |
529 | 537 | | |
530 | 538 | | |
531 | | - | |
532 | | - | |
| 539 | + | |
| 540 | + | |
533 | 541 | | |
534 | 542 | | |
535 | 543 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
22 | 47 | | |
0 commit comments