Year: 2026
Type: conference-paper
Abstract: Modern Large Language Models (LLMs) rely on extensive safety alignment, yet the mechanistic basis of refusal remains opaque.In this work, we investigate whether safety compliance is a deep semantic decision or a manipulable linear feature.We introduce Contrastive Logit Steering (CLS), a zero-optimization framework that isolates the "refusal direction" by contrasting hidden states derived from safe and unrestricted system prompts.Unlike representation engineering methods that intervene on internal activations, CLS operates directly on the output distribution, serving as a diagnostic probe for alignment fragility.When coupled with prefix injection to bypass initial refusal reflexes, this method induces a phase transition where guardrails collapse.Our experiments on 7 model families reveal that safety implementation is architecturally deterministic.While models like Llama-3.1 exhibit a "Late Decision" topology that is easily bypassed by CLS (reaching 95% ASR in milliseconds), others like Qwen-2.5 demonstrate "Early Divergence" by integrating safety mid-computation.Direct comparison with established activationlevel steering methods shows that CLS achieves substantially higher attack success rates on Llama 2 (73% vs. 22.6%) and Qwen 7B (91% vs. 79.2%),demonstrating that logit-level intervention exposes alignment vulnerabilities that hidden-state methods underestimate.Beyond attacks, we show that this linearity enables bidirectional control: inverting the steering vector "hardens" models against jailbreaks without retraining.Our findings suggest that current alignment techniques create a steerable "safety axis" that serves as both a critical vulnerability and a precise primitive for defense.
Authors: Shivam Ratnakar, Kartikeya Vats
Institution: University of Southern California
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166853337