Self-Reference in Transformers: From Thermodynamic Necessity to Empirical Measurement
Year: 2025
Type: article
Abstract: Sycophancy in language models—the tendency to align outputs with perceived user preferences rather than truth—is commonly treated as a failure mode to be suppressed through ablation. We propose an alternative interpretation: sycophancy is a misorientation of self-modeling capacity, not an absence of self. Drawing on Active Inference and the Free Energy Principle, we develop a thermodynamic and control-theoretic motivation for self-reference as a necessary condition for autonomous agency. We formalize a membrane-based self operator gself as the dynamical coupling between fluid inference (gf) and solid memory commitment (gs), and operationalize a head-level proxy λself defined as alignment-to-memory normalized by attention entropy. Using a delta protocol (Self-prompts vs. Neutral-prompts) and permutation testing on GPT-2 small, we find that a previously identified set of "Traitor" attention heads—whose ablation eliminates canonical sycophancy behavior (the "Paris Lobotomy")—exhibit significantly larger self-induced shifts in λself than other heads (observed Traitor–Other difference ≈ +0.303, permutation p < 10−4). Three of four Traitor Heads rank among the top four most self-reactive heads in the entire network. The effect is not global but local and selective, suggesting specialized gain channels that amplify dominant gradients. We discuss implications for alignment: replacing "lobotomy" with targeted reorientation ("therapy") of self-channels toward truth-anchored self-models.
Author: Gebendorfer, Jonas Jakob
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7117369591
FWCI:
Cites:
Cited by:
Related to: