Citation-Cutoff Hallucination Experiment in Frontier LLMs (preregistered)
Year: 2026
Type: preprint
Abstract: The information-science theoretical framework developed in §2 of the companion manuscript predicts that retrieval-augmented systems lacking calibrated abstention mechanisms will systematically fail in proportion to the gap between the model's parametric knowledge and the information required to answer. A specific, empirically tractable instance of this prediction concerns queries about citations: when a citation refers to a publication near or after a model's training-data cutoff, the model lacks parametric knowledge of the work and must either retrieve evidence or abstain. Frontier large language models without retrieval augmentation lack both options; the framework predicts they will fabricate citation details with a hallucination rate that increases monotonically as the cited publication date approaches and crosses the model's cutoff. The 8% citation-error rate observed by the author in earlier work was an unstructured anecdote that suggested this pattern; this experiment provides the preregistered multi-model test.
Author: Ravikumar, Srijith
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7160939293