Right Answer, Wrong Reason: Testing Source References and Responses to Evidence Changes
Year: 2026
Type: preprint
Abstract: An AI system can extract the right value from the wrong place in a document. When a bank relies on that value, checking the answer alone can miss the source error. This article proposes a separate source check followed by controlled changes to relevant and irrelevant evidence. The answer and its citation must then respond as the specified rule requires. The article sets out a bounded acceptance protocol for regulated document review. An answer fails if it is right by coincidence but rests on the wrong supporting field. Existing research on citations and behavioural testing motivates this design. A minimal date example shows the mechanism, and a constructed case shows what the controlled changes add to the source check. A proposed comparative experiment defines how added detection and false rejection are measured. The tests show observable behaviour under specified changes. They do not show a model's internal reasoning or its reliability in untested cases.
Author: Neuhaus, Daniel
Language: English
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7221517795
Cites:
Cited by:
Related to: