Skip to content

fix(game_eval): cap eval readiness wait below the server command timeout (#500)#503

Merged
dsarno merged 3 commits into
mainfrom
claude/issue-500-eval-ready-wait
Jun 1, 2026
Merged

fix(game_eval): cap eval readiness wait below the server command timeout (#500)#503
dsarno merged 3 commits into
mainfrom
claude/issue-500-eval-ready-wait

Conversation

@dsarno

@dsarno dsarno commented May 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Addresses the residual TimeoutError ~15s bucket in #500 (the larger of the two long-timeout slices the telemetry flagged post-2.5.10).

Root cause

The game_eval editor-side path (mcp_debugger_plugin.gd::request_game_eval_wait_then_eval) reused GAME_READY_WAIT_SEC = 20.0 — a wait tuned for the screenshot path, where a freshly launched game subprocess can take ~15s to boot and render a first frame.

But the eval path's total editor-side budget is ready_wait + eval_backstop (10s), and the server-side game_eval command timeout is 15s (handlers/editor.py::game_eval, mirrored by the dispatcher's 15000 ms game_eval budget). With a 20s readiness wait, a game that isn't running/ready makes the editor poll past the 15s server deadline, so the server gives up first with an opaque TimeoutError — the editor's actionable "Is the game actually running?" error never reaches the client.

This matches #500's hypothesis exactly: a hang path where "the eval request never reaches game_helper.gd, or no game is running," so only the outer server-side timeout catches it.

Fix

Give eval its own EVAL_READY_WAIT_SEC = 3.0 instead of inheriting the 20s screenshot wait:

  • Editor-side worst case is now 3s + 10s backstop = 13s, comfortably under the 15s server timeout, so the fast, actionable error always wins the race instead of an opaque 15s TimeoutError.
  • A game launched moments before the eval still has a 3s grace to register its mcp:hello; if it needs longer, the user gets a fast "is it running? start it and retry" message rather than a 15s hang.
  • The 20s GAME_READY_WAIT_SEC is unchanged for the screenshot path (which genuinely needs the longer boot window and whose 8s send timeout sits differently).

The TIMEOUT ORDERING comments are updated to document the new invariant: EVAL_READY_WAIT_SEC + eval backstop < server command timeout.

Scope / notes

  • The INTERNAL_ERROR ~10s bucket in game_eval: residual ~10–15s timeout failures not caught by the 8s eval bound (#488 follow-up) #500 is the editor-side eval backstop firing on a genuinely stuck/frozen eval (e.g. a backgrounded play-in-editor game with a frozen idle loop, or an infinite loop). That path is already actionable (it returns the "compiled and started running but never returned… likely an infinite loop or never-firing await" message), so this PR leaves it as-is — the opaque TimeoutError bucket was the higher-value target.
  • game_command (_wait_then_game_command) shares the same 20s-wait-vs-15s-command-timeout relationship and could get the same treatment, but it isn't in game_eval: residual ~10–15s timeout failures not caught by the 8s eval bound (#488 follow-up) #500's telemetry, so I've kept this change scoped to game_eval. Happy to extend if desired.
  • Final confirmation against the telemetry logs (which path each residual failure took) is still the issue's investigative ask and needs the maintainers' data — this PR closes the most likely opaque-timeout path.

Testing

  • script/ci-check-gdscriptAll GDScript files OK (parse-clean on the import pass).
  • Behavioral change is timing-only; logic paths (ready check, send, backstop, grace timer) are unchanged.

https://claude.ai/code/session_01Jq5X4ivngAf1N6r5UX2BVw


Generated by Claude Code

claude added 2 commits May 31, 2026 17:44
…500)

The game_eval not-ready path reused the 20s GAME_READY_WAIT_SEC tuned for
screenshots. That wait exceeds the 15s server-side game_eval command timeout,
so when a game isn't running/ready the editor polled for up to 20s while the
server gave up first with an opaque ~15s TimeoutError — the editor's
actionable "Is the game actually running?" error never reached the client.
This is #500's residual ~15s TimeoutError bucket.

Give eval its own EVAL_READY_WAIT_SEC (3s). Editor-side worst case is now
3s wait + 10s backstop = 13s, comfortably under the 15s server timeout, so the
fast actionable error always wins the race. The 20s wait stays for screenshots
(a freshly launched game needs longer to render a first frame).

Refs #500.

https://claude.ai/code/session_01Jq5X4ivngAf1N6r5UX2BVw
The actionable hint pointed at a nonexistent 'play_scene' op; the MCP tool
that runs the game is project_run.

https://claude.ai/code/session_01Jq5X4ivngAf1N6r5UX2BVw
@codecov

codecov Bot commented May 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts the editor-side game_eval “wait for game readiness” window so it can’t outlast the 15s server/dispatcher game_eval timeout, preventing opaque ~15s TimeoutError failures and allowing the editor to return a faster, actionable “game not running/ready” error instead.

Changes:

  • Introduces a dedicated EVAL_READY_WAIT_SEC = 3.0 (separate from the 20s screenshot readiness wait).
  • Updates game_eval’s readiness polling deadline to use EVAL_READY_WAIT_SEC.
  • Expands the inline timeout-ordering documentation around game_eval to capture the new invariant (ready_wait + backstop < 15s).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +37 to +38
## MUST stay below the server-side game_eval command timeout (15s, see
## handlers/editor.py::game_eval / dispatcher.gd's 15000ms game_eval budget).
Comment on lines +434 to +435
_send_error(connection, request_id, ErrorCodes.INTERNAL_ERROR,
"Game-side autoload never registered its debugger capture within %ds. Is the game actually running?" % int(GAME_READY_WAIT_SEC))
"Game-side autoload never registered its debugger capture within %ds. Is the game actually running? Start it with project_run / the editor's Play button, then retry." % int(EVAL_READY_WAIT_SEC))
…ady error (#500)

Addresses Copilot review on #503:
- The EVAL_READY_WAIT_SEC doc referenced 'handlers/editor.py' and lumped
  dispatcher.gd's 15000ms budget as 'server-side'. Correct to the full path
  (src/godot_ai/handlers/editor.py) and note dispatcher.gd's budget is
  editor/plugin-side; the 15s ceiling is enforced at both layers.
- The not-ready error was optimized for 'game not running' but the same
  symptom occurs when the _mcp_game_helper autoload is missing/disabled.
  Point users to Project Settings → Autoload too (mirrors the screenshot path).

https://claude.ai/code/session_01Jq5X4ivngAf1N6r5UX2BVw
@dsarno dsarno merged commit c728323 into main Jun 1, 2026
15 checks passed
@dsarno dsarno deleted the claude/issue-500-eval-ready-wait branch June 1, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants