Get started
Work

When Correct Answers Carry Invalid State: Evaluating Explicit State Carriers in Stateful LLM Agents

Year: 2026
Type: preprint
Abstract: Stateful LLM agents often expose bounded state for later computation. A correct final answer does not establish that this state is safe, sufficient for continuation, or aligned with the answer. We operationalize this acceptance question with a task-relative, field-sensitive, representation-normalized contract for explicit state carriers. Deterministic adapters map native summary, visible-carry, and typed-state representations into a common claimed-state layer; the contract then evaluates parseability, actionable-state presence, exclusion safety, operational sufficiency, and answer–state consistency. We apply the contract to 7,200 frozen outputs over 40 tasks, five model-provider conditions, six state-transfer protocols, two state budgets, and three runs. Dedicated final-answer success occurs on 696 rows (9.67%); intrinsic state validity on 191 (2.65%); answer-aligned validity on 180 (2.50%); and joint answer–carrier success on 173 (2.40%). Thus, only 24.86% of answer-correct outputs carry answer-aligned valid state, leaving a 75.14% silent-state-failure rate. In the three protocols with estimable valid and invalid strata, valid state is associated with higher next-step success, with task-cluster risk differences of 35.42–72.97 percentage points. Because the separately implemented executor shares task semantics with the validator, these contrasts provide criterion-related operational evidence rather than an unrelated external ground truth. Under this benchmark contract, final-answer accuracy is insufficient when downstream computation consumes the exposed state. All revised scores are replayed offline from saved outputs without model regeneration.
DOI: 10.5281/zenodo.22720975
Language: English
Open Access status: green
OpenAlex ID: W7212404667
Cites:
Cited by:
Related to: