In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning
Year: 2026
Type: preprint
Abstract: Retrieval-augmented generation (RAG) grounds a language model in retrieved documents, which reduces hallucination but creates a new attack surface: if retrieved text is tampered with, the model may repeat the falsehood. We study how much a small quantized model, Llama 3.1 8B, degrades when a fraction of its retrieved context is poisoned. Three corruption strategies are tested, entity swap, number swap, and negation, each applied to zero, one, two, or three of the three retrieved passages, over a factorial sweep of 588 runs on a fact-checking task built from FEVER. Accuracy falls from 77.9% on clean context to 43.5% when all three passages are corrupted. Entity swap is the most damaging in terms of answers flipped from correct to wrong, and number-based corruption shows a sharp threshold once poisoned passages form the majority. The model rarely invents new falsehoods; instead its dominant reaction is to abstain, and hallucination actually drops under attack. These results quantify a practical weakness of RAG and identify abstention, not fabrication, as the main behaviour to plan for.
Source: Qeios
DOI: 10.32388/7hxkqe
Author: Iliano Fasolino
Institution: University of Milan
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7202216485