Commit e435377
fix: prevent infinite whitespace in grammar rules
Fixes #93
The whitespace rule `ws ::= ([ \t\n]+)` allows unlimited consecutive
whitespace characters. At higher temperatures (e.g., 0.8), this can cause
the model to generate endless tabs/spaces, resulting in:
- Token exhaustion
- Invalid JSON output
- ~10% failure rate in function calling
Changed to `ws ::= ([ \t\n]?)` which:
- Allows zero or one whitespace character
- Prevents infinite loops while maintaining JSON validity
- More restrictive than `{1}` (allows optional whitespace)
Tested with Mistral 7B Instruct v0.2.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>1 parent 26848ef commit e435377
File tree
1 file changed
+1
-1
lines changed- src/llama_cpp_agent/gbnf_grammar_generator
1 file changed
+1
-1
lines changedLines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
802 | 802 | | |
803 | 803 | | |
804 | 804 | | |
805 | | - | |
| 805 | + | |
806 | 806 | | |
807 | 807 | | |
808 | 808 | | |
| |||
0 commit comments