Work

Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models

Year: 2026

Type: conference-paper

Abstract: Removing a specific individual's information from multimodal large language models (MLLMs) is often needed after deployment, but existing methods rely on a retain set, which is hardest to obtain at that point, and rebuilding it recreates the privacy exposure that unlearning aims to remove.Forgetting from the forget set alone instead damages the shared visuallanguage computation, harming perception.We cast retain-free unlearning as a localization problem: causal tracing, weight transplant, and Fisher overlap all point to early-to-mid decoder MLPs as the layers where identity information is stored and, unlike other module families, can be modified without substantially disrupting vision.We turn this into Pathway-Aware Visualattribute Anchoring (PAVA), which confines updates to these layers and pairs a forget loss with a visual-attribute anchor that preserves imagegrounded behavior by distilling the model's own pre-unlearning answers from the forget images alone.On MLLMU-Bench and ReMem, PAVA gives the strongest forget-retain tradeoff among forget-set-only methods and remains competitive with retain-based baselines.

Source: Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)

DOI: 10.18653/2026.findings-emnlp.1139

Authors: Kangwook Ko, Jaehyuk Jang, Wonjun Lee, Hee-Seon Kim, Changick Kim

Institution: Korea Advanced Institute of Science and Technology

Language: English

Topic: Privacy-Preserving Technologies in Data

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funders: Ministry of Science and ICT, South Korea, Institute for Information and Communications Technology Promotion

OpenAlex ID: W7204911089