Work

Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation

Year: 2026

Type: conference-paper

Abstract: Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering.However, existing multimodal RAG models are primarily designed for single-image or closed-document settings and exhibit limited accuracy in realistic multi-image scenarios.Moreover, processing numerous retrieved images incurs substantial computational overhead from irrelevant visual tokens.To address these challenges, we introduce DocLon-gRAG, a large-scale dataset of 343K questionanswer pairs, each associated with an average of 37.4 retrieved images to reflect authentic RAG workflows.Building on this dataset, we propose Doc-REFRAG, a question-guided framework that compresses visual tokens into coarse chunks and selectively expands questionrelevant ones via a lightweight RL-based selector.Experiments on six benchmarks show that Doc-REFRAG outperforms eleven strong baselines, achieving state-of-the-art accuracy with significantly lower inference latency.Our resources are available at https://github. com/Collab-Gen/Doc-REFRAG.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/2026.emnlp-main.377