Evidence-State Drift: A Claim-Level Framework and Replay Benchmark for Web-Connected LLM Agents
Year: 2026
Type: article
Abstract: This paper introduces Evidence-State Drift, a claim-level framework for monitoring changes in evidence used by web-connected large language model (LLM) agents over repeated retrieval events. The framework represents evidence using a normalized claim, evidence span, source identity, retrieval and publication timestamps, content fingerprint, and source-authority label. It aims to distinguish material changes in claim-associated evidence from non-material modifications, such as formatting changes, that do not alter the monitored claim. The study evaluates the proposed representation using a longitudinal replay benchmark containing 67 claims and 871 observation events, including 47 oracle-defined material changes. Within this benchmark, the claim-state detector achieved an F1 score of 0.989 against a predefined deterministic change oracle, with no false positives among 824 oracle-defined non-material events. These results indicate consistency with the benchmark’s predefined change rules. They do not establish independent semantic validation or improved factual accuracy in deployed LLM agents. Independent human evaluation and end-to-end agent testing remain areas for future work. This research contributes to the study of evidence monitoring, information freshness, data provenance, and reliability in web-connected AI systems.
Author: Kinger, Jai
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7222899354
FWCI:
Cites:
Cited by:
Related to: