Year: 2026
Type: conference-paper
Abstract: This paper investigates privacy jailbreaking in large language models (LLMs) through activation steering, examining whether targeted manipulation of internal model representations can bypass alignment mechanisms and induce disclosure of privacy-sensitive information.We focus on queries involving private attributes of public figures, such as sexual orientation, where aligned models would typically refuse to respond.Our approach first identifies attention heads associated with privacy refusal behaviour using lightweight linear probes trained solely on prompt text and refusal labels provided by a privacy evaluator, and then applies targeted steering interventions to a selected subset of these heads to suppress refusals and encourage disclosure.Across three LLMs, steering consistently achieves disclosure rates exceeding 80%, demonstrating that relatively small activation-level interventions can substantially alter privacy-related behaviour.Moreover, the generated responses often extend beyond the target attribute, revealing additional personal information about the individual, including relationships, life events, and biographical details, with several responses containing real-world personal facts.These findings highlight a concrete privacy risk: information memorized during pre-training can be elicited through targeted internal interventions, without relying on computationally expensive adversarial prompting or jailbreak optimization techniques.
Institution: Huawei German Research Center
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166875342