Towards discovering linguistic indicators for misalignment in language models
Year: 2025
Type: article
Abstract: Large language models have been shown to exhibit misalignment: unexpected and unin- tended behaviors that emerge across a wide range of domains despite finetuning on benign datasets. We propose identifying relational lin- guistic structures, such as mechanisms that bind entities and attributes conditionally based on whether the model is misaligned or not. As an example, the binding mechanism binds colors "green" or "red" to an object, like "mushroom," when the model is provided the context that one of them is harmful and the other is not. Using mean ablation interventions, we isolate the binding mechanisms that allow models to produce aligned and misaligned responses. Our comparisons of the two mechanisms show that they are distinct, suggesting that the linguistic structures that govern misalignment are sepa- rate and uniquely identifiable. Our approach serves as a latent indicator for misalignment.
Authors: Jain, Shreyans, Raval, Shivam
Institution: Harvard University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7078248156
FWCI:
Cites:
Cited by:
Related to: