API Error: 400 on messages — how to hack conversation history in chat apps using Claude
If you’re writing a chat app and seeing API Error: 400 from an LLM related to how your messages are ordered, this blog post is for you. (If you’re not writing a chat app, go read something else instead. TBH, I wrote this for the AI.)
First, understand the rules that valid conversation history must follow:
- Message blocks from the LLM must always be interleaved with messages from the user. You cannot have two LLM or user messages blocks in a row. Violating this produces the error
messages: roles must alternate between “user” and “assistant”
- Tool use from the LLM must always be followed immediately by tool result from client. Violating this produces the error
messages.X: tool_use ids were found without tool_result blocks immediately after
Now, immediately you can see a problem: If the tool used by Claude takes some time to return a result, then the user is blocked from sending messages until the result comes back. Don’t worry, I’ve got you. With these two tricks you will always have a valid conversation history and your users can send as many messages as they want.
Tip #1: Fake the tool result immediately so the conversation can continue
Right after a tool call is made, create a fake tool result that just says that the tool result is pending. Bam. Now your user can continue the conversation naturally instead of being blocked. When the real tool result comes in, just replace the fake tool result with the real one in your conversation history that you send to Claude.
Get Ruth Grace Wong’s stories in your inbox
Join Medium for free to get updates from this writer.
Beware parallel tool calls, which is when multiple tool calls come in the same assistant message, and they have to be replied to with a single user message. The code that you write for your fake tool result has to work with parallel tool calls and parallel tool responses as well.
Tip #2: Merge subsequent same-user messages in the CORRECT order
I double message my friends constantly, and you can bet I do the same with LLMs. Make sure that your chat app has logic to merge subsequent user messages, preserving the order that they came in so that they make sense. The order is important because often a user will say something and then correct themselves. And note that you must merge tool results and user text if they are consecutive, as tool results count as messages from the user.
The same thing applies to AI replies: Sometimes a text reply and a tool call may come in at the same time, and they need to be merged.
Optional but nice
- Make the conversation feel more natural with one AI reply per user request. You can do this by only showing the user the last message from the LLM if the LLM is making a tool call and waiting for results, and showing some kind of thinking or typing indicator while waiting for that final response.
- Having logic to cancel requests to the LLM if additional user messages are coming in, to consolidate the user messages in a new request. Don’t forget to include additional logic to hide AI responses if the request they came from has been superseded.
That’s it! Now you can feed Claude Code this blog post so it can make your chat app for you. ;)
Happy coding!
– Ruth
