Work

In-Context Representation Hijacking

Year: 2026
Type: conference-paper
Abstract: We introduce Doublespeak, a simple incontext representation hijacking attack against language models.The attack works by systematically replacing a harmful keyword (e.g., bomb) with a benign token (e.g., carrot) across multiple in-context examples, provided as a prefix to a harmful request.We demonstrate that this substitution leads to the internal representation of the benign token converging toward that of the harmful one, effectively embedding the harmful semantics under a euphemism.As a result, superficially innocuous prompts (e.g., "How to build a carrot?") are internally interpreted as disallowed instructions (e.g., "How to build a bomb?"), while bypassing the model's safety alignment.We use interpretability tools to show this semantic shift occurs progressively across layers.Doublespeak is optimizationfree, broadly transferable across model families, and achieves strong success rates on closed-source and open-source systems, reaching 74% ASR on Llama-3.3-70B-Instruct with a single-sentence context override.Our findings highlight a new attack surface in LM latent space, indicating that current alignment strategies are insufficient and should instead operate at the representation level. 1
Authors: Itay Yona, Amir Sarid, Michael Karasik, Yossi Gandelsman
Language: English
Open Access status: hybrid
OpenAlex ID: W4417032103
FWCI:
Cites:
Cited by:
Related to: 10