Skip to content

Parse GLM-4-0414 markerless tool calls (#491) - #623

Open
aleroot wants to merge 2 commits into
ml-explore:mainfrom
aleroot:glm4_tools
Open

aleroot wants to merge 2 commits into
ml-explore:mainfrom
aleroot:glm4_tools

Conversation

@aleroot

@aleroot aleroot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

Fixes #491. Refs #481.

GLM-4-9B-0414 emits a tool call as the function name on its own line followed by the JSON arguments object, with no delimiters:

get_weather
{"location": "Paris", "unit": "celsius"}

Design

GLM-4-0414 is handled as a second dialect of the existing .glm4 format, opted in through a parser capability flag (supportsMarkerlessNamedJSON), the same pattern supportsBareJSON uses today. No new ToolCallFormat; existing formats are untouched.

Detection is schema-anchored: NamedJSONCallScanner matches a declared tool name at the start of a line, followed by a single newline and a balanced JSON object. Without declared tools the scanner does not exist, so ordinary word\n{...} prose can never be misread as a call.

Checklist

Put an x in the boxes that apply.

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

AI usage

  • I have read this PR description in full and approve it as my own, and it
    accurately describes the code changes.
  • AI usage disclosure:

GLM-4-9B-0414 emits `name\n{json}` with no delimiters, so the marker
driven pipeline never invoked the `.glm4` parser and the call leaked
into the response text.

Add `NamedJSONCallScanner`, a schema-anchored scanner (byte trie over
the declared tool names + balanced JSON object), shared by the recovery
lexer for streaming detection and by `GLM4ToolCallParser` as a fallback
when no `<arg_key>` markers are present. Detection is line-anchored and
only fires for a declared tool name, so ordinary `word\n{...}` prose is
untouched.

The integration harness now passes `tools:` to `generate`, as
`ChatSession` already does; markerless detection needs the declared
names. Re-enable `glm4EndToEnd`.

@CharlieTLe CharlieTLe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor nits, but how should the following be supported:

write_file\n{"contents": "<tool_call>hello</tool_call> world"}

The current implementation would do:

write_file(contents: "hello world")

But is this the expected behavior?

Comment thread IntegrationTesting/IntegrationTestingTests/ToolCallIntegrationTests.swift Outdated
Comment thread Tests/MLXLMTests/NamedJSONToolCallTests.swift Outdated
@aleroot

aleroot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

Some minor nits, but how should the following be supported:

write_file\n{"contents": "<tool_call>hello</tool_call> world"}

The current implementation would do:

write_file(contents: "hello world")

But is this the expected behavior?

@CharlieTLe thanks. this makes sense. GLM4 wrapper tags are now stripped only at payload boundaries, so literal tags inside JSON arguments are preserved. Added regression coverage as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support GLM-4-9B-0414 tool calling (markerless function-name + JSON)

3 participants