Entropy Is Not Enough: Output-Distribution Signals for Detecting Context–Memory Conflicts in Retrieval-Augmented Generation
Year: 2026
Type: preprint
Abstract: Retrieval-augmented generation (RAG) systems can be misled when retrieved context contradicts what the model knows, and a cheap inference-time conflict detector would let costly arbitration be applied selectively. We ask whether the output distribution of a language model carries a usable conflict signal. We compare entropy-based signals with identity-aware signals (Jensen–Shannon divergence, argmax disagreement, probability mass moved away from the closed-book answer, likelihood of the closed-book answer under the context). In experiments with Qwen2.5-3B/7B-Instruct on TriviaQA (type-matched wrong-answer contexts) and CounterFact, with a neutral-context control, entropy shift is near chance in three of four settings (AUC 0.46–0.54) and at most 0.65 in the fourth, whereas identity-aware signals reach AUC 0.84–0.95 on CounterFact and 0.51–0.76 on TriviaQA. A conditional analysis shows these signals largely detect that the model was swayed by the context. Downstream, gating on identity-aware signals matches or exceeds always-arbitrating within sampling error while arbitrating only 32–57% of instances, whereas entropy-based gating loses poison resistance; arbitration prompt wording matters more than the gate. Results are limited to two small models, 83–100 test questions per setting, and synthetic conflicts. Code and results are included in the archive. This is a preprint and has not been peer reviewed.
Author: Hanzala, Muhammad
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225633378
Cites:
Cited by:
Related to: