Year: 2026
Type: conference-paper
Abstract: Recently, large language models (LLMs) have demonstrated impressive performance but still suffer from hallucinations.Model editing has been proposed as a means to correct factual inaccuracies.A challenging scenario is sequential model editing (SME), which aims to rectify errors continuously, rather than as a one-time task.During SME, the general capabilities of LLMs can be negatively affected due to the introduction of new parameters.In this paper, we propose a queue-based self-correction framework, QueueEDIT, that not only enhances SME performance by addressing long-sequence dependencies but also mitigates the impact of parameter bias on the general capabilities of LLMs.Specifically, we first introduce a structural mapping editing loss to map editing triplets to knowledge-sensitive neurons within the Transformer layers.We then store the located parameters for each piece of edited knowledge in a queue and dynamically align previously edited parameters.At each edit, we select parameters in the queue that are most relevant to the currently located parameters to determine whether the knowledge associated with previous edits requires realignment.Irrelevant parameters in the queue are frozen, and we update the LLM with the parameters at the head of the queue to ensure they do not harm general capabilities.Experiments show that QueueEDIT significantly outperforms strong baselines across various SME settings, while maintaining competitive performance in single-turn editing.The resulting LLMs also preserve high performance on general NLP tasks throughout the SME process.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166801074