RAG-Critic: Leveraging Automated Critic-Guided Agentic Workflow for Retrieval Augmented Generation
Year: 2025
Type: conference-paper
Abstract: Retrieval-augmented generation (RAG) has emerged as a pivotal technology in natural language processing, owing to its efficacy in generating factual content.However, its informative inputs and complex paradigms often lead to a greater variety of errors.Consequently, achieving automated on-policy assessment and error-oriented correction remains an unresolved issue.In this paper, we propose RAG-Critic, a novel framework that leverages a critic-guided agentic workflow to improve RAG capabilities autonomously.Specifically, we initially design a data-driven error mining pipeline to establish a hierarchical RAG error system.Based on this system, we progressively align an errorcritic model using a coarse-to-fine training objective, which automatically provides finegrained error feedback.Finally, we design a critic-guided agentic RAG workflow that customizes executor-based solution flows based on the error-critic model's feedback, facilitating an error-driven self-correction process.Experimental results across seven RAG-related datasets confirm the effectiveness of RAG-Critic, while qualitative analysis offers practical insights for achieving reliable RAG systems.Our dataset and code are available at https: //github.com/RUC-NLPIR/RAG-Critic.
Institution: Renmin University of China
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G1899978318, G228125052, G7230045735
OpenAlex ID: W4412886707