Skip to content

Fix incorrect assumption when parsing tool_calls[].function.arguments - #1904

Open
LxYuan0420 wants to merge 1 commit into
ml-explore:mainfrom
LxYuan0420:fix/server-tool-call-dict-arguments
Open

LxYuan0420 wants to merge 1 commit into
ml-explore:mainfrom
LxYuan0420:fix/server-tool-call-dict-arguments

Conversation

@LxYuan0420

Copy link
Copy Markdown
Contributor

This PR is:

  • to fix the incorrect assumption that tool_calls[].function.arguments is always a JSON string
  • to parse function.arguments with json.loads() only when it is a string, and leave dict untouched

Reproduce command:

$ mlx_lm.server \
  --model mlx-community/functiongemma-270m-it-4bit \
  --port 8081


$ curl -i http://127.0.0.1:8081/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"default_model","messages":[{"role":"user","content":"what is 2+3?"},{"role":"assistant","content":null,"tool_calls":
  [{"type":"function","id":"call_123","function":{"name":"add","arguments":{"a":2,"b":3}}}]},
  {"role":"tool","name":"add","tool_call_id":"call_123","content":"5"}],"max_tokens":16}'

Before fix:

HTTP/1.0 404 Not Found
{"error": "the JSON object must be str, bytes or bytearray, not dict"}
Screenshot 2026-09-20 at 2 21 16 PM

After fix:

HTTP/1.0 200 OK
chat.completion response
Screenshot 2026-09-20 at 2 20 49 PM

TL;DR: If function.arguments is a string, parse it; if it’s already a dict, leave it untouched.

  • ☑️ I understand it is strictly prohibited to use AI to write PR description

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant