Skip to content

partialParse in _vendor/partial-json-parser crashes on invalid JSON escape sequences from model output #986

@HazAT

Description

@HazAT

Bug

When a model emits tool call JSON containing characters like unescaped backslashes (e.g. PHP namespaces App\Http\Middleware\HandleAppearance), the partialParse function in _vendor/partial-json-parser/parser.{js,mjs} throws:

Bad escaped character in JSON at position 80 (line 1 column 81)

This crashes the MessageStream during input_json_delta accumulation (line 501 in MessageStream.mjs), killing the entire streaming response.

Root Cause

The generate() function reconstructs JSON from tokens but does not sanitize string values. The tokenizer correctly captures escape sequences like \H from App\Http, but generate() outputs them verbatim back into a JSON string:

// generate() outputs: "use App\Http\Middleware" — \H and \M are invalid JSON escapes
output += ' + token.value + ';

JSON.parse() then rejects \H, \M, \A etc. since only \" \\ \/ \b \f \n \r \t \uXXXX are valid JSON escape sequences.

Fix

In the generate() function, sanitize string token values by doubling backslashes that precede invalid escape characters:

case 'string':
    // Repair invalid JSON escapes (e.g. \H from PHP namespaces)
    let sanitized = token.value.replace(/\\([^"\\\/bfnrtu])/g, \\$1);
    output += " + sanitized + ";
    break;

This turns \H\\H, \M\\M etc., producing valid JSON that JSON.parse() accepts.

Additionally, the partialParse() call in MessageStream.{js,mjs} should be wrapped in a try/catch so a parse failure during streaming does not crash the entire response.

Reproduction

import { partialParse } from "./_vendor/partial-json-parser/parser.mjs";

// Simulates model outputting PHP namespace backslashes in edit tool JSON
const input = '{"edits": [{"oldText": "use App\\Http\\Middleware\\HandleAppearance;", "newText": ""}]}';
partialParse(input); // throws: Bad escaped character in JSON

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions