Work

QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation

Year: 2026

Type: conference-paper

Abstract: Dynamic Retrieval-Augmented Generation adaptively determines when to retrieve during generation to mitigate hallucinations in large language models (LLMs).However, existing methods rely on model-internal signals (e.g., logits, entropy), which are fundamentally unreliable because LLMs are typically ill-calibrated and often exhibit high confidence in erroneous outputs.We propose QuCo-RAG, which shifts from subjective confidence to objective statistics computed from pre-training data.Our method quantifies uncertainty through two stages: (1) before generation, we identify low-frequency entities indicating long-tail knowledge gaps; (2) during generation, we verify entity co-occurrence in the pre-training corpus, where zero co-occurrence often signals hallucination risk.Both stages leverage Infini-gram for millisecond-latency queries over 4 trillion tokens, triggering retrieval when uncertainty is high.Experiments on multi-hop QA benchmarks show QuCo-RAG achieves EM gains of 5-12 points over state-of-the-art baselines with OLMo-2 models, and transfers effectively to models with undisclosed pre-training data (Llama-3, Qwen2.5, GPT-4.1/5-chat),improving EM by up to 14 points.Generalization to long-form generation and biomedical QA further validates the robustness of our paradigm.These results establish corpus-grounded verification as a principled, practically model-agnostic paradigm for dynamic RAG 1 .

Source: Findings of the Association for Computational Linguistics: ACL (Findings of ACL)

DOI: 10.18653/v1/2026.findings-acl.812

Authors: Dehai Min, Kailin Zhang, Tongtong Wu, Lu Cheng

Institutions: University of Illinois Chicago, New York University, Monash University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid