Year: 2026
Type: conference-paper
Abstract: Emergent misalignment, where fine-tuning on narrow, non-harmful tasks induces harmful behaviors, poses a key challenge for AI safety in LLMs.Despite growing empirical evidence, its underlying mechanism remains unclear.To uncover the reason behind this phenomenon, we propose a geometric account based on the geometry of feature superposition.Because features are encoded in overlapping representations, fine-tuning that amplifies a target feature also unintentionally strengthens nearby harmful features in accordance with their similarity.We give a simple gradient-level derivation of this effect and empirically test it in multiple LLMs (Gemma-2 2B/9B/27B, LLaMA-3.1 8B, gpt-oss 20B).Using sparse autoencoders (SAEs), we identify features tied to misalignment-inducing data and to harmful behaviors, and show that they are geometrically closer to each other than features derived from non-inducing data.This trend generalizes across domains (e.g., health, career, legal advice).Finally, we show that a geometry-aware approach-filtering training samples closest to toxic features-reduces misalignment by 34.5%, substantially outperforming random removal and achieving comparable or slightly lower misalignment than LLM-as-ajudge-based filtering.Our study links emergent misalignment to feature superposition, providing a basis for understanding and mitigating this phenomenon.This paper analyzes datasets and model outputs that include offensive language.
Institutions: The University of Tokyo, Google DeepMind (United Kingdom)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166847658