May the Memory Be With You: Efficient and Infinitely Updatable State for Large Language Models
Year: 2025
Type: conference-paper
Abstract: Large language models (LLMs) excel at natural language tasks but lack persistent state management for personalized and adaptive interactions. We propose a framework that endows these models with stateful capabilities by combining retrieval-augmented generation (RAG) and low-rank adaptation (LoRA). Our approach recasts the LLM as an editable component that retains hierarchical knowledge, analogous to deferred merge operations in log-structured merge (LSM) trees. The system integrates short-term context with long-term memory by storing accumulated context in a retrieval system while periodically training and combining lightweight LoRA adapters. Preliminary evaluations demonstrate improved state retention and query performance compared to both standard LLMs and RAG-augmented models, supporting our vision for scalable, stateful AI systems.
Authors: Excel Chukwu, Laurent Bindschaedler
Institution: Max Planck Institute for Software Systems
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4409060959