When Conversations Cannot Resume: An Empirical Study of Persistent-State Failures in LLM Applications
Year: 2026
Type: preprint
Abstract: LLM applications save conversations so that users can reload and resume them. We study what happens when a saved conversation cannot be resumed. A stuck conversation's stored record never reaches a finished state. Every later request built from a bricked conversation's stored history is rejected, because the history, or the code that replays it, breaks the message rules. Under a protocol written before any issue was classified, we analysed 275 GitHub issues (261 root-cause clusters, 128 repositories), labelled by an LLM pipeline checked against a blind human rater. We reproduced 12 defects in 10 applications, before and, where one exists, after the fix, and confirmed the rejections on the real Anthropic and OpenAI APIs. The main finding is that, in our reproductions, bricked conversations were mostly broken not by the agent loop but by the code around it: in 5 of 8, an extension, middleware, plugin, compaction step or command dispatcher saves a message between a tool call and its result. Fixes tend to repair the history when it is read back rather than prevent the bad write, and repair or fix code itself went wrong in three reproductions. Rules tested against the real API agreed with Anthropic on 120 of 121 held-out histories, and on 80 of 80 after one revision; our OpenAI rules agreed with OpenAI on 161 of 177, and every miss was a duplicate result. The history check in LangGraph's prebuilt agent, which asks only whether each call has a result somewhere, caught only 38–42% of the histories Anthropic rejected. The rules also differ by API: OpenAI's Responses API accepts the interleaving that Anthropic and Chat Completions reject. Because defects on the read path leave no trace in storage, saved conversations should be checked both at rest and on the outgoing request. Finally, we used the mechanism to search 15 current open-source applications and frameworks, and found six previously unreported defects in five of them. The five that brick conversations were each confirmed on the real API; in each, a message is saved while a tool call is still open, and three of these five sit in or next to code added by an earlier fix.
Author: Thapa, Shirshak Jung
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Keywords: LLM applications, LangGraph, OpenAI API
Open Access status: green
OpenAlex ID: W7235371741
