v1/chat/completions to deepseek r1 distill model: output with [TOOL_REQUEST] and [END_TOOL_REQUEST] should return as tool_calls

[zutuanwang]

Occasionally, output with [TOOL_REQUEST] and [END_TOOL_REQUEST] return as normal text
{"created":1739000946,"usage":{"completion_tokens":335,"prompt_tokens":1570,"total_tokens":1905},"model":"deepseek-r1-distill-qwen-32b-mlx","id":"chatcmpl-aw1gme3n1gdgs4oa173a9","choices":[{"finish_reason":"stop","index":0,"message":{"role":"assistant","content":"<think>\n好的，我现在需要帮助用户查询今天的天气情况。用户的问题是“今天广州的天气如何”。首先，我要确定用户的需求是什么，他们想知道广州今天的天气状况。\n\n接下来，我需要查看可用的工具。根据提供的工具列表，有一个“weather_GET”工具，可以用来获取当前天气信息。这个工具需要一个参数“zoneId”，即IANA时区数据库标识符。广州的时区是“Asia/Shanghai”，所以我需要使用这个参数来调用工具。\n\n然后，我需要按照指定的格式发送工具请求。正确的格式是[TOOL_REQUEST]标签内包含JSON对象，指定工具名称和参数。因此，我会发送以下请求：\n\n[TOOL_REQUEST]\n{\"name\": \"weather_GET\", \"arguments\": {\"zoneId\": \"Asia/Shanghai\"}}\n[END_TOOL_REQUEST]\n\n发送请求后，我会等待工具返回结果。假设工具返回了天气数据，我需要解析这些数据，并以用户友好的方式呈现出来。例如，如果返回的数据包括温度、天气状况和风速等信息，我会将这些内容整理成一段自然的中文回复。\n\n如果工具返回的结果不完整或有误，我需要检查是否正确调用了工具，并确保参数无误。如果问题依旧存在，可能需要联系技术支持或寻找其他途径获取天气信息。\n\n最后，确保回复内容准确无误，并且语言自然流畅，符合用户的需求。这样，用户就能清楚地了解今天广州的天气情况了。\n</think>\n\n[TOOL_REQUEST]\n{\"name\": \"weather_GET\", \"arguments\": {\"zoneId\": \"Asia/Shanghai\"}}\n[END_TOOL_REQUEST]"}}],"system_fingerprint":"deepseek-r1-distill-qwen-14b@8bit","object":"chat.completion"}

it should return as tool_calls like this
{"created":1738921631,"usage":{"completion_tokens":278,"prompt_tokens":1563,"total_tokens":1841},"model":"deepseek-r1-distill-qwen-14b@8bit","id":"chatcmpl-khk6ajtwjgbdbm61vz2f","choices":[{"finish_reason":"tool_calls","index":0,"message":{"role":"assistant","tool_calls":[{"function":{"name":"weather_GET","arguments":"{\"zoneId\":\"Asia/Shanghai\"}"},"id":"243734958","type":"function"}]}}],"system_fingerprint":"deepseek-r1-distill-qwen-14b@8bit","object":"chat.completion"}

[GrahamTheCoder]

^{Edit: Was seen on non mlx engine too, so has been moved from there}
As a partial workaround in agentic setups, you can set [TOOL_REQUEST] or <tool_call> as a stop string (whichever your model uses), then prompt the model to just return a tool call when that happens. Though as someone mentioned in #453, the LLM may use TOOL_REQUEST within thinking tags which will break that approach.
Issue #453 seems to be in a similar area around parsing TOOL_REQUEST in the presence of <think> tags
Let me know if there's config anywhere where I can set the tool call start/end tag