When Sources Disagree: Accuracy-Preserving Batch Verification and the Limits of Source-Age Screening in Retrieval-Augmented Generation
Year: 2026
Type: preprint
Abstract: Retrieval-augmented generation (RAG) can attach citations that contradict a generated claim or no longer reflect current facts. We revisit an experimentally evaluated citation-verification design using SciFact for semantic contradiction and a small, independently reviewed FreshQA case study for temporal change. On 200 balanced SciFact SUPPORT/CONTRADICT cases, a cross-encoder MiniLM-NLI baseline achieved 72.0% accuracy and 0.7144 macro-F1. The originally proposed negation-first shortcut reduced accuracy to 66.5%, despite reducing model invocations; consequently, we do not recommend that shortcut as an accuracy-preserving gate. In a separate five-repeat CPU throughput benchmark, length-sorted batch inference (batch size four) reduced total processing time from 51.779 s to 46.317 s (10.55%) while reproducing 100% of baseline predictions. Historical-source review identified four genuine FreshQA answer changes and one unchanged control. Applied to these five 148-day-old sources, the fixed 90-day age rule flagged all five: it alerted on all four corroborated changes and also on the one unchanged control (four true alerts, one false alert). Source age therefore indicates a need for re-verification, not proof of factual obsolescence. The findings support accuracy-preserving throughput optimization and motivate content-aware temporal checks; neither online latency improvement nor generalizable staleness classification is established.
Source: Research Square
Institution: Tshwane University of Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225730343