Optimal tool call sequences for common AI agent workflows. Each recipe shows the exact tools in order, key parameters, what to check in responses, and common pitfalls.
Complete flow from page open to submit.
geometra_connect({
"pageUrl": "https://boards.greenhouse.io/company/jobs/123",
"returnForms": true,
"includeOptions": true,
"returnPageModel": true
})Check: formCount > 0 in response. Note the schemaId and formId for later. If captcha is present in the page model, see Recipe 4.
geometra_fill_form({
"formId": "fm:1.0",
"valuesByLabel": {
"First Name": "Jane",
"Last Name": "Doe",
"Email": "jane@example.com",
"Phone": "555-0100",
"LinkedIn URL": "https://linkedin.com/in/janedoe",
"How did you hear about us?": "Referral"
},
"skipPreFilled": false,
"verifyFills": true
})Check: completed: true, errorCount: 0. If verification.mismatches is non-empty, a field rejected input (see Recipe 7). If stoppedAt is present, resume from resumeFromIndex (see Recipe 6).
geometra_upload_files({
"paths": ["/Users/you/resume.pdf"],
"fieldLabel": "Resume"
})Check: Response confirms file count. If the error says "no file input found by label", retry with strategy: "hidden".
geometra_wait_for_resume_parse({
"text": "Parsing",
"timeoutMs": 15000
})Check: Success means the parsing banner disappeared. If it times out, the site may not show a banner -- continue anyway.
geometra_form_schema({
"formId": "fm:1.0",
"includeOptions": true
})Check: Compare field values. Resume parsing may have filled some fields. Only fill remaining gaps.
geometra_click({
"name": "Submit Application",
"role": "button",
"waitFor": {
"text": "Application submitted",
"timeoutMs": 10000
}
})Check: postWait confirms the success message appeared. If the button is disabled, check final.invalidFields from the last fill response for missing required fields.
- Greenhouse sometimes uses custom dropdowns for location/department. If
fill_formerrors on a choice field with "no option found", fall back togeometra_pick_listbox_option(Recipe 3). - The Submit button may say "Submit", "Submit Application", or "Apply" -- use
geometra_find_actionif unsure. - Some Greenhouse forms split into sections behind "Continue" buttons -- that makes it multi-page (see Recipe 2).
Workday applications span several pages (Personal Info, Experience, Voluntary Disclosures, etc.).
geometra_connect({
"pageUrl": "https://company.wd5.myworkdayjobs.com/careers/job/Location/Title_ID/apply",
"returnForms": true,
"returnPageModel": true,
"includeOptions": true
})Check: Page model archetypes -- Workday often shows as ["form", "wizard"]. Note formCount.
geometra_fill_form({
"formId": "fm:1.0",
"valuesByLabel": {
"First Name": "Jane",
"Last Name": "Doe",
"Email Address": "jane@example.com",
"Phone Number": "555-0100",
"Country": "United States"
},
"skipPreFilled": true,
"failOnInvalid": true,
"includeSteps": true
})Check: completed: true. If failOnInvalid triggered, inspect final.invalidFields for missing required fields.
geometra_click({
"name": "Next",
"role": "button"
})geometra_wait_for_navigation({
"timeoutMs": 15000
})Check: navigated: true. The response includes formCount for the new page and a page model summary. If navigated: false, the page may validate inline -- check for error alerts.
geometra_workflow_state({})Check: pageCount should show the completed page(s). totalInvalidRemaining should be 0 for filled pages.
geometra_form_schema({
"includeOptions": true
})geometra_fill_form({
"valuesByLabel": {
"How did you hear about us?": "LinkedIn",
"Are you legally authorized to work in the United States?": "Yes"
},
"skipPreFilled": true
})Continue the click Next -> wait_for_navigation -> form_schema -> fill_form loop until you reach the review/submit page.
geometra_click({
"name": "Submit",
"role": "button",
"waitFor": {
"text": "submitted",
"timeoutMs": 20000
}
})- Workday pages can be slow. Use
timeoutMs: 15000or higher onwait_for_navigation. - Country/state fields are often custom listboxes. If
fill_formfails on those, usegeometra_pick_listbox_optionwith aquery. - Workday sometimes requires scrolling to reveal fields. If a field fill fails with "not visible", call
geometra_revealwith the field name first. - The "Next" button text varies: "Next", "Continue", "Save and Continue". Use
geometra_find_actionto locate it.
Custom dropdowns (React Select, Headless UI, Radix, Ashby-style) don't work with native <select> -- use geometra_pick_listbox_option.
geometra_form_schema({
"includeOptions": true
})Check: Look for fields with type: "combobox" or type: "listbox". Note the label and available options.
geometra_pick_listbox_option({
"fieldLabel": "Location",
"label": "New York, NY",
"query": "New York"
})Parameters:
fieldLabel-- the form field label from the schema (e.g., "Location", "Department")label-- the option text to select (supports fuzzy/alias matching: "US" matches "United States")query-- text to type into a searchable combobox before selecting (triggers remote search for async options)
Check: Response includes readback with the confirmed field value. If the pick fails, the error payload includes visibleOptions showing what options are actually available -- use one of those labels to retry.
geometra_pick_listbox_option({
"fieldLabel": "Location",
"label": "New York, New York, United States",
"exact": true
})Only for native HTML <select> elements. If form_schema shows type: "select", use:
geometra_select_option({
"x": 400,
"y": 300,
"label": "United States"
})- If the dropdown loads options asynchronously (common for location pickers), always pass
queryto trigger the search, and increasetimeoutMsto 8000+. - Some dropdowns open on focus, not click.
pick_listbox_optionhandles this automatically viafieldLabel. - If the same label appears in multiple dropdowns (e.g., two "Location" fields), use
sectionTextorcontextTextto disambiguate.
Detect CAPTCHAs early and bail with a clear message rather than wasting tool calls.
geometra_connect({
"pageUrl": "https://jobs.example.com/apply/123",
"returnPageModel": true
})Check the response for captcha in the page model.
The page model includes a captcha field when a CAPTCHA is detected:
{
"captcha": {
"kind": "recaptcha-v2",
"visible": true
}
}If captcha is present and visible: true, stop automation and return a message to the human:
"This page requires a CAPTCHA (reCAPTCHA v2). Please complete the CAPTCHA manually, then tell me to continue."
geometra_page_model({})Check: captcha should be absent or visible: false. Then proceed with form_schema and fill_form.
If captcha.kind is "recaptcha-v3" or visible: false, the CAPTCHA is invisible/automatic. Proceed normally -- it will resolve on submit. Only bail for visible challenge CAPTCHAs.
- CAPTCHAs can appear after clicking Submit, not just on page load. After any submit click, check the page model again.
- Some sites use hCaptcha or Turnstile instead of reCAPTCHA -- the
kindfield will reflect the type.
Upload a resume, wait for the ATS to finish parsing, then fill remaining fields without overwriting parsed data.
geometra_upload_files({
"paths": ["/Users/you/resume.pdf"],
"fieldLabel": "Resume/CV",
"timeoutMs": 10000
})Check: Confirm upload succeeded. If fieldLabel fails, try without it (auto-detects the file input) or use strategy: "hidden".
geometra_wait_for_resume_parse({
"text": "Parsing",
"timeoutMs": 20000
})Tune text per site:
- Greenhouse:
"Parsing" - Lever:
"Uploading"or"Processing" - Ashby:
"Parsing your resume" - Workday: often no banner (skip this step)
geometra_form_schema({
"includeOptions": true
})Check: Fields that were empty before may now have values from the parsed resume (name, email, phone, etc.).
geometra_fill_form({
"valuesByLabel": {
"First Name": "Jane",
"Last Name": "Doe",
"Email": "jane@example.com",
"Phone": "555-0100",
"LinkedIn URL": "https://linkedin.com/in/janedoe"
},
"skipPreFilled": true
})Check: skippedPreFilled in the response shows how many fields were skipped because parsing already filled them correctly.
- Upload the resume BEFORE filling text fields. Many ATS platforms overwrite all fields when parsing completes, erasing your previous fills.
- If there's no parsing banner visible (some sites parse instantly or silently), skip step 2. Just wait a beat with
geometra_wait_foron a field you expect to become non-empty:geometra_wait_for({ "name": "First Name", "role": "textbox", "timeoutMs": 5000 }) - Cover letters are separate uploads. Use a second
upload_filescall withfieldLabel: "Cover Letter".
When geometra_fill_form or geometra_fill_fields stops on an error, check the suggestion and retry with the recommended tool.
geometra_fill_form({
"valuesByLabel": {
"First Name": "Jane",
"Location": "NYC",
"Resume": "/Users/you/resume.pdf"
},
"stopOnError": true,
"includeSteps": true
})If a field fails, the response includes stoppedAt and a suggestion on the failing step:
{
"completed": false,
"stoppedAt": 1,
"resumeFromIndex": 2,
"steps": [
{ "index": 0, "kind": "text", "ok": true },
{
"index": 1,
"kind": "choice",
"ok": false,
"error": "No option matching 'NYC' found in select",
"suggestion": "Try geometra_pick_listbox_option with fieldLabel=\"Location\" and label=\"NYC\" for custom dropdowns."
}
]
}geometra_pick_listbox_option({
"fieldLabel": "Location",
"label": "NYC",
"query": "New York"
})geometra_fill_form({
"valuesByLabel": {
"Resume": "/Users/you/resume.pdf"
},
"resumeFromIndex": 2
})| Error | Suggestion | Action |
|---|---|---|
| "No option matching X" on a choice field | Use geometra_pick_listbox_option |
Switch to listbox tool with query |
| "Timeout" on a choice field | Increase timeoutMs or use pick_listbox_option with query |
Retry with longer timeout |
| "No textbox found for label X" | Field may be a non-standard control | Use geometra_query to find the field, then geometra_click + geometra_type |
| "No file input found by label" | Use geometra_upload_files with strategy: "hidden" |
Retry with hidden strategy or click coordinates |
| "Action timed out" (generic) | Page may still be loading | Call geometra_wait_for with a loading indicator, then retry |
- Do not set
stopOnError: falsefor critical forms. It will skip errored fields silently, and you may submit an incomplete application. - When resuming with
resumeFromIndex, the form schema may have changed (e.g., conditional fields appeared). Re-fetch withgeometra_form_schemaif the resume attempt also fails.
Use verifyFills to catch fields where the ATS silently rejected or transformed input (e.g., autocomplete replaced your text, phone format changed).
geometra_fill_form({
"valuesByLabel": {
"First Name": "Jane",
"Last Name": "Doe",
"Phone": "(555) 010-0100",
"City": "NYC"
},
"verifyFills": true
})The response includes a verification object:
{
"completed": true,
"verification": {
"verified": 4,
"mismatches": [
{
"fieldLabel": "Phone",
"expected": "(555) 010-0100",
"actual": "5550100100",
"fieldId": "phone_field"
},
{
"fieldLabel": "City",
"expected": "NYC",
"actual": "",
"fieldId": "city_field"
}
]
}
}Phone reformatted: Usually harmless. If actual contains the same digits, the site reformatted to its preferred format. No action needed.
City empty/wrong: The field rejected the input (possibly a listbox, not a textbox). Fix it:
geometra_pick_listbox_option({
"fieldLabel": "City",
"label": "New York",
"query": "New York"
})The response also includes minConfidence (0.0 to 1.0):
{
"completed": true,
"minConfidence": 0.65,
"fieldCount": 8
}Confidence thresholds:
>= 0.9-- high confidence, likely correct0.7 - 0.9-- moderate, worth a quick check< 0.7-- low confidence. Some fields matched by fuzzy label. Review the fill plan withincludeSteps: trueand checkmatchMethodon each step (e.g.,"exact"vs"fuzzy"vs"alias")
If minConfidence < 0.7 or mismatches are non-empty and the actual value is empty/wrong:
"Some fields may not have filled correctly. Please review: Phone (reformatted to 5550100100), City (could not fill -- may need manual selection). Confidence: 0.65."
verifyFillsonly checks text and choice fields, not toggles or file uploads. Verify those separately withgeometra_queryif needed.- Autocomplete can replace your value after a brief delay. If you suspect this, add a
geometra_wait_forwithvalueset to your expected text before verifying. - Some sites format phone numbers, dates, or SSNs on blur. A mismatched
actualis not always an error -- compare the semantic content, not the exact string.
Geometra MCP has two connection modes. Every recipe above uses the proxy path (Chromium + Playwright). Recipes 8-10 below use the native path (direct WebSocket to a Geometra server).
| Native | Proxy | |
|---|---|---|
| Connect with | url: "ws://localhost:3100" |
pageUrl: "https://..." |
| Browser needed | No | Yes (Chromium auto-spawned) |
| Form-fill tools | Click + type + key only | fill_form, fill_fields, pick_listbox_option, etc. |
| Best for | Geometra-native apps, agent-first UX, multi-agent | Existing websites, ATS automation |
| Startup time | Instant | 2-5s (browser launch) |
Choose native when you're building a new Geometra app and want agents as first-class users. Choose proxy when automating an existing website.
See NATIVE_MCP_GUIDE.md for the full tool compatibility matrix.
Connect to a running Geometra server (no browser, no proxy) and interact through semantic tools.
Prerequisite: Start the CRUD demo server:
cd demos/mcp-native-crud && npm run servergeometra_connect({
"url": "ws://localhost:3100"
})Check: Connection confirms with session info. No browser is launched.
geometra_page_model({})Returns buttons ("Add Task", filter buttons), a task list, status text. The agent now understands the app structure.
geometra_click({ "role": "button", "name": "Add Task" })Opens the edit form with a title input and priority selector.
geometra_click({ "role": "textbox" })
geometra_type({ "text": "Review pull request" })The server receives each keystroke, updates its signal state, and broadcasts the new frame.
geometra_click({ "role": "button", "name": "Save" })geometra_query({ "role": "status" })Check: Response text should be "Task created: Review pull request".
geometra_snapshot({})Shows the updated task list with the new task visible.
- Always call
geometra_clickon the textbox beforegeometra_type-- the server routes key events to the focused element. - If
geometra_typehas no effect, verify withgeometra_snapshotthat the input is focused. - Use
geometra_query({ role: "status" })to verify actions rather than relying on the snapshot alone.
A complete add / filter / edit / delete workflow with verification between each step.
Prerequisite: The CRUD demo server is running on ws://localhost:3100.
geometra_connect({ "url": "ws://localhost:3100" })
geometra_click({ "role": "button", "name": "Add Task" })
geometra_click({ "role": "textbox" })
geometra_type({ "text": "Deploy to staging" })
geometra_click({ "role": "button", "name": "Save" })
geometra_query({ "role": "status" })Check: Status = "Task created: Deploy to staging"
geometra_click({ "role": "button", "name": "Filter Active" })
geometra_snapshot({})Check: Only uncompleted tasks appear.
geometra_click({ "role": "checkbox", "name": "Deploy to staging" })
geometra_query({ "role": "status" })Check: Status = "Task completed: Deploy to staging". The task disappears from the active filter.
geometra_click({ "role": "button", "name": "Filter Done" })
geometra_snapshot({})Check: "Deploy to staging" now appears in the done list.
geometra_click({ "role": "button", "name": "Filter All" })
geometra_click({ "role": "button", "name": "Edit Deploy to staging" })The edit form opens with the title pre-filled.
geometra_click({ "role": "textbox" })
geometra_key({ "key": "Meta+a" })
geometra_type({ "text": "Deploy to production" })
geometra_click({ "role": "button", "name": "Save" })
geometra_query({ "role": "status" })Check: Status = "Task updated: Deploy to production"
geometra_click({ "role": "button", "name": "Delete Deploy to production" })
geometra_query({ "role": "status" })Check: Status = "Task deleted: Deploy to production"
- Edit and Delete buttons include the task title in their
ariaLabel(e.g., "Edit Deploy to staging"). Use the full name to target the right task when multiple tasks exist. - After toggling a checkbox, the task may disappear if a filter is active. Switch to "All" first if you need to interact with it further.
Two agents connect to the same native server and see each other's changes in real time.
geometra_connect({ "url": "ws://localhost:3100" })
geometra_click({ "role": "button", "name": "Add Task" })
geometra_click({ "role": "textbox" })
geometra_type({ "text": "Write unit tests" })
geometra_click({ "role": "button", "name": "Save" })geometra_connect({ "url": "ws://localhost:3100" })
geometra_snapshot({})Agent B sees the task that Agent A just created. Now B marks it done:
geometra_click({ "role": "checkbox", "name": "Write unit tests" })geometra_snapshot({})Agent A sees the task is now checked. No polling needed -- snapshot reads the latest frame, and the server broadcasts every update() to all clients.
Each geometra_connect creates an independent MCP session, but they share the same app state on the server. When any client (human or agent) triggers an action, the server:
- Updates its signals
- Calls
server.update() - Recomputes layout via Yoga
- Broadcasts the new frame to all connected WebSocket clients
MCP sessions receive the updated frame automatically. The next tool call from any agent reads the latest state.
- Concurrent writes to the same field are last-write-wins. There is no conflict resolution -- the last agent to trigger an action determines the state.
- If two agents click the same button simultaneously, both actions fire. Design your state transitions to be idempotent where possible.
When the flow is fill values → click Submit → wait for success/nav, prefer geometra_submit_form over three separate tool calls. It auto-connects, fills via the form-schema resolver, clicks the submit target, and waits — one round trip.
geometra_submit_form({
"pageUrl": "https://boards.greenhouse.io/company/jobs/123",
"isolated": true,
"valuesByLabel": {
"First Name": "Jane",
"Last Name": "Doe",
"Email": "jane@example.com"
},
"submit": { "role": "button", "name": "Submit application" },
"waitFor": { "role": "dialog", "name": "Application submitted", "timeoutMs": 15000 }
})Check:
completed: true— the whole chain landed.navigated: truewithafterUrl— the submit caused a navigation.waitFor: { present: true, matchCount: 1 }— the success condition fired.final.invalidCount— residual invalid fields after the submit wait resolves.
When values were already written by a prior call (e.g. resume parsing) and you only need to submit:
geometra_submit_form({
"skipFill": true,
"submit": { "role": "button", "name": "Submit application" },
"waitFor": { "role": "dialog", "name": "Application submitted" }
})submitdefaults to{ role: 'button', name: 'Submit' }— only omit it when that generic filter picks the right button.waitForis optional. When omitted, the tool returns as soon assendClickis acknowledged.failOnInvalid: trueturns any residualinvalidCountinto an error result (useful when a form re-renders with new validation after submit).
Semantic actions (geometra_click, geometra_fill_fields, geometra_fill_form) now emit fallback metadata when their happy-path resolution failed and a recovery attempt succeeded. geometra_run_actions aggregates step-level fallbacks into a top-level fallbacks array so the signal is visible even when includeSteps: false.
// geometra_click result fragment
{
"at": { "x": 320, "y": 480 },
"target": { "role": "button", "name": "Submit" },
"fallback": { "attempted": true, "used": true, "reason": "relaxed-visibility", "attempts": 2 }
}Reasons:
revision-retry— the initial semantic resolve missed because the UI tree was still settling after a navigation. Waiting for one revision tick let the target appear.relaxed-visibility— the caller requiredfullyVisible: truebut the element could only be revealed as partially visible (sticky headers, tall inputs, overlays).
When every fallback phase was attempted but none recovered the target, the tool returns an error result whose text is a structured JSON payload:
{
"error": "No elements found matching { \"role\": \"button\", \"name\": \"Submit\" }",
"fallback": {
"attempted": true,
"used": false,
"reasonsTried": ["revision-retry", "relaxed-visibility"],
"attempts": 3
}
}Parse the error as JSON when you need the telemetry. Plain-text errors still come back unchanged when no fallback was attempted (explicit coordinates, empty filter).
// geometra_fill_form result fragment (includeSteps: false)
{
"execution": "sequential",
"fallback": { "attempted": true, "used": true, "reason": "batched-threw", "attempts": 2 }
}Reasons:
batched-unavailable— the batched proxy path returned a recoverable error; the sequential loop picked up.batched-threw— the batched sendFillFields threw (e.g. proxy reported an unsupported message type).batched-invalid-readback— batched ack said "ok" but the a11y readback showed fields still invalid.
{
"completed": true,
"stepCount": 3,
"successCount": 3,
"fallbacks": [
{ "stepIndex": 0, "type": "fill_fields", "attempted": true, "used": true, "reason": "batched-unavailable", "attempts": 2 },
{ "stepIndex": 2, "type": "click", "attempted": true, "used": true, "reason": "revision-retry", "attempts": 2 }
]
}- Agents: ignore
fallbackfor flow control — the action still succeeded. Don't branch on it; don't retry. The only exception is treating repeated fallbacks across a single session as a hint that the page might need a different strategy (e.g.isolated: trueon the next connect). - Operators: aggregate
fallback.reasoncounts in logs to prioritize native fixes. A spike inbatched-invalid-readbackfor a given site means the batched proxy path needs tightening; a spike inrelaxed-visibilitysuggests a sticky element that should be revealed natively.
Common errors and how to resolve them.
The server isn't running, or you're using the wrong port/URL.
- Native: Verify the server process is running and check the port. Default is
ws://localhost:3100. - Proxy: Verify the URL is reachable from your machine. Try
curl -I <url>first.
You must call geometra_connect before using any other tool. Each MCP session starts disconnected.
You called a proxy-only tool (fill_form, fill_fields, set_checked, select_option, pick_listbox_option, upload_files, wheel) on a native Geometra server. Use click + type + key instead. See the tool compatibility matrix in NATIVE_MCP_GUIDE.md.
The element doesn't exist, isn't visible, or has a different name than expected.
- Run
geometra_snapshot({ "view": "full" })to see the complete tree - Check element roles with
geometra_query({ "role": "button" })to list all buttons - If using
name, try a substring:geometra_query({ "text": "Submit" })instead of an exact name match
If geometra_snapshot shows old state after an action, the server may not have called server.update(). This is a server-side bug in the app being automated.
For proxy connections, the page may still be loading. Use geometra_wait_for with a condition that indicates the update completed.
The target element isn't focused. Click the input/textbox first:
geometra_click({ "role": "textbox", "name": "Title" })
geometra_type({ "text": "hello" })On native servers, the input component must have an onKeyDown handler wired up -- see NATIVE_MCP_GUIDE.md section 5.