Right Number, Forged Pedigree: A Preregistered Minimal Study of False Auditability in LLM Answers over XBRL Knowledge Graphs
Year: 2026
Type: preprint
Abstract: Preregistered analysis plan, frozen before any call to a real language model. Files: plan, configuration, frozen corpus (sha256 inside), dry-run cost record, offline gate results. Graph-grounded LLM systems are often called auditable because their answers cite graph nodes. A citation only supports an audit if it is valid: the cited nodes exist and the answer can be recomputed from them. This study measures two things on real models. Study A asks how often a model gives a numerically correct answer whose citation is not valid (false auditability). Study B asks whether a model acting as an auditor accepts a correct answer whose cited provenance has been forged. The study is deliberately small (30 firm-years, 240 items, six open-weight models) and is designed to answer one question each, with offline controls that fix what a perfect and a blind system would score.
Author: Chahed, Oussama
Institution: Université Paris Dauphine-PSL
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Keywords: auditability, financial reporting
Open Access status: green
OpenAlex ID: W7235378285
