Add screenshot-testing recipe doc + local game-capture diagnostic by dsarno · Pull Request #414 · hi-godot/godot-ai

dsarno · 2026-05-08T14:16:45Z

Context

Surfaced by a user who kept hitting

INTERNAL_ERROR: Game-side autoload never registered its debugger capture within 20s.
Is the game actually running? Check Project Settings → Autoload for _mcp_game_helper.

while having an AI client test a runtime UI option (a new main-menu entry).
The 20s timeout in mcp_debugger_plugin.gd:202-204 fires whenever the
editor sends mcp:take_screenshot but never sees mcp:hello from the
game-side autoload — and there are several sharp edges that make AI agents
hit it repeatedly:

mcp:hello is only honored when _game_run_active=true, i.e. when the
current play cycle was started by project_run. Manual F5 plays beacon
is explicitly ignored (mcp_debugger_plugin.gd:113-116), so any AI that
reaches for editor_screenshot(source="game") without first calling
project_run will time out.
editor_state already exposes game_capture_ready (set by _setup_session
/ begin_game_run and flipped true on mcp:hello), but agents don't
always know to poll it — they sleep instead, miss the window, and time out.
The pre-flight question "is _mcp_game_helper actually persisted to
project.godot?" requires either eyeballing the file or calling
autoload_manage(op="list"), which agents skip.
Multi-editor / multi-worktree setups can land the screenshot tool call on
a session whose game isn't running.

Changes

docs/screenshot-testing.md — agent-facing recipe:

source selection table (when to use viewport vs game vs cinematic).
The canonical recipe: project_run(autosave=False) → poll
editor_state.game_capture_ready → editor_screenshot(source="game") →
project_stop.
Pre-flight checklist for the timeout + a decision tree by symptom.
Notes on preferring state assertions (node_get_properties, print()
from the game forwarded over mcp:log_batch) when the assertion isn't
truly visual — screenshots are the slowest, flakiest assertion surface.

script/local-game-capture-diag — developer-facing companion to the
existing script/ci-game-capture-smoke:

Works against whatever scene is currently open (no capture_smoke.tscn
fixture required) so it's usable for "is this broken in MY project?".
Dumps the in-memory autoload list (autoload_manage) AND the on-disk
[autoload] section of project.godot (filesystem_manage read_text)
before touching the game, so the autoload-missing case is visible.
Polls game_capture_ready deterministically — same readiness signal the
CI smoke uses, no sleep-and-pray.
On failure, dumps recent plugin logs and game logs (logs_read and
logs_read source="game") so a crashed boot vs. a missing autoload vs.
a wrong session is distinguishable.
--run / --no-screenshot / --keep-running flags for the common
diagnostic shapes.

Test plan

python3 -c "import ast; ast.parse(open('script/local-game-capture-diag').read())" — done locally
ruff check script/local-game-capture-diag — done locally
script/local-game-capture-diag --help renders argparse correctly — done locally
Run script/local-game-capture-diag --run against an open editor on a project with _mcp_game_helper registered → expect PASS: end-to-end game capture bridge works.
Remove _mcp_game_helper from project.godot and rerun → expect the autoload-missing path to print the project.godot section with the entry absent and the actionable HINT.
Skim docs/screenshot-testing.md rendered on GitHub for formatting.

No code paths in src/ or plugin/ change — this is docs + a diagnostic
script.

🤖 Generated with Claude Code

Generated by Claude Code

docs/screenshot-testing.md captures the agent-facing recipe for editor_screenshot — when to pick viewport vs game vs cinematic, the project_run → poll game_capture_ready → screenshot pattern, the pre-flight checklist for the "autoload never registered its debugger capture within 20s" timeout, and a decision tree for diagnosing it. Also calls out preferring state assertions (node_get_properties, game print() forwarded over mcp:log_batch) over pixel screenshots when the test isn't truly visual. script/local-game-capture-diag is the developer-facing companion to ci-game-capture-smoke: works against whatever scene is currently open (no fixture scene required), dumps the autoload list and project.godot [autoload] section before touching the game, polls game_capture_ready deterministically, and on failure dumps recent plugin + game logs so the failure mode is visible (autoload missing vs game crashed during boot vs F5-launched-not-project_run vs wrong session active).

codecov · 2026-05-08T14:19:55Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copilot

Pull request overview

This PR adds guidance and a local diagnostic script to make editor_screenshot(source="game") more reliable and debuggable, especially for the common “game-side autoload never registered…” timeout scenario.

Changes:

Add an agent-facing documentation recipe for screenshot-driven testing, including readiness polling via editor_state.game_capture_ready.
Add a developer-facing local diagnostic script to validate the editor↔game screenshot bridge and dump autoload + logs for failure triage.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

File	Description
`script/local-game-capture-diag`	New local diagnostic to connect to MCP, check autoload persistence, poll `game_capture_ready`, and attempt a game screenshot.
`docs/screenshot-testing.md`	New documentation describing when to use each screenshot source and a recommended run/poll/screenshot/stop workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        if we_started_run and not args.keep_running:
+            try:
+                _tool_call(args.url, sid, "project_stop", {}, 999, timeout=10)
+            except Exception as exc:


+    5. call editor_screenshot(source="game") and print PNG dimensions +
+       byte size — does NOT assert on pixel colors, so it works against
+       any scene


+    print(f"Connecting to {args.url}")
+    try:
+        sid = _initialize(args.url)
+        sess = _wait_for_session(args.url, sid)
+    except (urllib.error.URLError, RuntimeError) as exc:
+        print(f"\nSetup failed: {exc}", file=sys.stderr)
+        print(
+            "Is the Godot editor open with the plugin enabled?\n"
+            "Is the MCP server running on the expected port?",
+            file=sys.stderr,
+        )
+        return 2
+    print(f"Session: {sess.get('active_session_id') or sess}")
+


+2. project_run(mode="current",      -> autosave=False so MCP scene mutations stay in memory
+              autosave=False)
+3. poll editor_state every ~500ms   -> wait until is_playing=true AND game_capture_ready=true
+4. (interact: input_send / node_set_property / call_method as needed)


+4. (interact: input_send / node_set_property / call_method as needed)
+5. editor_screenshot(source="game", -> request the capture
+                     max_resolution=1280)
+6. project_stop                     -> always stop the run when done


+  → Did you `project_stop` and forget to `project_run` again? Each new run
+    rotates a token; the readiness flag is reset on `begin_game_run()`.


+    parser.add_argument(
+        "--keep-running",
+        action="store_true",
+        help="Skip project_stop on exit (default is to stop the run we started).",


…ims, --session flag - project_stop is not a top-level MCP tool — stop is project_manage(op="stop"). Updated both doc references (recipe step 6 + decision-tree mention) and the script's cleanup call + --keep-running help text. - input_send / call_method aren't registered tools. Genericized recipe step 4 to point at the actual interaction surface (node_set_property, batch_execute, ui_manage, theme_manage) without prescribing a specific tool. - Module docstring promised "PNG dimensions + byte size" but only printed byte length. Added _png_dimensions(): parses width/height from the IHDR chunk (zero-dep), printed alongside byte count on success. - Added --session <hint> flag for the multi-editor routing case the doc explicitly calls out as a timeout cause. Calls session_activate before any other tool, prints the resolved session, fails fast on no-match.

dsarno requested a review from Copilot May 8, 2026 14:22

Copilot started reviewing on behalf of dsarno May 8, 2026 14:23 View session

Copilot AI reviewed May 8, 2026

View reviewed changes

dsarno merged commit d7cbe52 into main May 11, 2026
11 checks passed

dsarno deleted the claude/debug-screenshot-testing-CmwyY branch May 11, 2026 15:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add screenshot-testing recipe doc + local game-capture diagnostic#414

Add screenshot-testing recipe doc + local game-capture diagnostic#414
dsarno merged 2 commits into
mainfrom
claude/debug-screenshot-testing-CmwyY

dsarno commented May 8, 2026

Uh oh!

codecov Bot commented May 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		→ Did you `project_stop` and forget to `project_run` again? Each new run
		rotates a token; the readiness flag is reset on `begin_game_run()`.

Conversation

dsarno commented May 8, 2026

Context

Changes

Test plan

Uh oh!

codecov Bot commented May 8, 2026

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants