LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples

Year: 2026

Type: conference-paper

Abstract: Large language models (LLMs) possess vast knowledge acquired from extensive training corpora, but they often cannot remove specific pieces of information when needed, which makes it hard to handle privacy, bias mitigation, and knowledge correction.Traditional model unlearning approaches require computationally expensive fine-tuning or direct weight editing, making them impractical for real-world deployment.In this work, we introduce LoRA-based Unlearning with Negative Examples (LUNE 🌙 ), a lightweight framework that performs negative-only unlearning by updating only lowrank adapters while freezing the backbone, thereby localizing edits and avoiding disruptive global changes.Leveraging Low-Rank Adaptation (LoRA), LUNE targets intermediate representations to suppress (or replace) requested knowledge with an order-of-magnitude lower compute and memory than full fine-tuning or direct weight editing.Extensive experiments on multiple factual unlearning tasks show that LUNE: (I) achieves effectiveness comparable to full fine-tuning and memory-editing methods, and (II) reduces computational cost by about an order of magnitude.

Source: Findings of the Association for Computational Linguistics: ACL (Findings of ACL)

DOI: 10.18653/v1/2026.findings-acl.816

Authors: Yezi Liu, Hanning Chen, Wenjun Huang, Yang Ni, Mohsen Imani

Institutions: University of California, Irvine, Purdue University Northwest

Language: English

Topic: Privacy-Preserving Technologies in Data

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, Semiconductor Research Corporation, Cisco Systems, Defense Advanced Research Projects Agency, Office of Naval Research, U.S. Military Academy, Army Research Office

OpenAlex ID: W7166807441