Response from Anthropic models is sometimes truncated

[benthamite]

Please update gptel first -- errors are often fixed by the time they're reported.

• I have updated gptel to the latest commit and tested that the issue still exists

Bug Description

Responses from Anthropic models, such as Claude Sonnet 3.5 and 3.7, sometimes get truncated. I attach a file with an example (see line ending with (insert (or (). I can confirm that this never happens with any of the other models I tried, Including o1, o3-mini, various Gemini models, Grok 2, and others.

make-tlon-ai-fix-markdown-format-more-robust.org.zip

I have experienced this intermittently for several months, so it is not due to any recent changes.

Steps to Reproduce

1. Download and uncompress the attached file, then open the org file.
2. M-x eval-expression RET (goto-char 16976). This will place point where it was just prior to sending the request that returned the truncated response.
3. Confirm that the active model is an Anthropic model; if not, select one, such as Claude:claude-3-7-sonnet-20250219.
4. M-x gptel-send

Additional Context

GNU Emacs 30.0.93 (build 2, aarch64-apple-darwin24.3.0, NS appkit-2575.40 Version 15.3 (Build 24D60)) of 2025-01-29
macOS Sequoia Version 15.3.1 (24D70)

[lispy-ai]

For what it's worth, I have seen this too. Highly intermittent but seems to be specific to these models.

[benthamite]

As a workaround, I have created this Yasnippet:

# -*- mode: snippet -*-
# name: truncated-code
# key: tc
# contributor: Pablo Stafforini
# --
It seems your code was truncated. Please provide the remainder, without including what you have already written.

So I just hit tc TAB at the end of the buffer and the model will resume just where it left. However, this doubles the cost of the request, takes more time, and is generally annoying.

[karthink]

Could you run gptel with gptel-log-level set to 'info? The next time this happens, you can compare the buffer to the log and let me know.

Another question of interest is if this also happens when you turn off streaming responses.