Unifying Retrieval and Generation: A Survey on Retrieval-Augmented Generation in NLP
Year: 2025
Type: preprint
Abstract: Retrieval-Augmented Generation (RAG) has emerged as a transformative approach in the field of natural language processing (NLP), combining the capabilities of information retrieval with generative models to tackle a wide range of language understanding and generation tasks. Traditional generative models, while proficient in generating fluent and coherent responses, often struggle with knowledge grounding, especially when dealing with domain-specific or long-tail queries. Retrieval-Augmented Generation addresses this limitation by leveraging external knowledge sources, such as large document corpora, to improve the factual accuracy and contextual relevance of the generated content. In RAG, the retrieval component fetches pertinent information from these knowledge sources, which is then fused with the input query and used to guide the generative model in producing responses that are both informed and coherent. This survey provides an extensive review of the RAG paradigm, focusing on its key components: the retrieval mechanism, the generative model, and the integration between them. We delve into various retrieval techniques employed in RAG systems, including traditional sparse retrieval methods (e.g., BM25) and modern dense retrieval methods that utilize neural embeddings, such as those based on transformer architectures. The paper compares these methods in terms of their retrieval performance, efficiency, and scalability, highlighting the trade-offs between retrieval accuracy and computational cost. We also discuss the challenges and complexities in the fusion of retrieved knowledge with the generation process, emphasizing the importance of ensuring coherence, relevance, and factual correctness in the generated outputs. In addition to the foundational techniques, we explore various challenges faced by RAG systems. These challenges include the retrieval of relevant and high-quality documents, the need for effective fusion strategies that combine the retrieved knowledge with the input query, and the prevention of hallucinations-where the model generates plausiblesounding but incorrect information. We also address the computational complexities involved in training and fine-tuning RAG models, including the difficulties of joint optimization between the retrieval and generative components. Furthermore, ethical considerations, such as bias in the retrieval sources and the generated outputs, are discussed, underscoring the need for fairness, transparency, and responsible AI practices in the deployment of RAG systems.
Source: HAL
Institution: Lancaster University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4415273139