Work

MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation

Year: 2025

Type: conference-paper

Abstract: Processing long contexts presents a significant challenge for large language models (LLMs). While recent advancements allow LLMs to handle much longer contexts than before (e.g., 32K or 128K tokens), it is computationally expensive and can still be insufficient for many applications. Retrieval-Augmented Generation (RAG) is considered a promising strategy to address this problem. However, conventional RAG methods face inherent limitations because of two underlying requirements: 1) explicitly stated queries, and 2) well-structured knowledge. These conditions, however, do not hold in general long-context processing tasks.

DOI: 10.1145/3696410.3714805

Authors: Hongjin Qian, Zheng Liu, Peitian Zhang, Kelong Mao, Defu Lian, Zhicheng Dou, Tiejun Huang

Institutions: Peking University, Beijing Academy of Artificial Intelligence, Hong Kong Polytechnic University, Renmin University of China, University of Science and Technology of China

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Funders: National Natural Science Foundation of China, Ministry of Education, India

Awards: G1899978318, G228125052

OpenAlex ID: W4410089611

FWCI: 27.67

Cites: 26

Cited by: 28

Related to: 10