Year: 2024
Type: conference-paper
Abstract: Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents.Although there are diverse jailbreak attack strategies, there is no unified understanding on why some methods succeed and others fail.This paper explores the behavior of harmful and harmless prompts in the LLM's representation space to investigate the intrinsic properties of successful jailbreak attacks.We hypothesize that successful attacks share some similar properties: They are effective in moving the representation of the harmful prompt towards the direction to the harmless prompts.We leverage hidden representations into the objective of existing jailbreak attacks to move the attacks along the acceptance direction, and conduct experiments to validate the above hypothesis using the proposed objective.We hope this study provides new insights into understanding how LLMs understand harmfulness information.1 * These authors contributed equally to this work.1 Our code is available at https://github.com/ yuplin2333/representation-space-jailbreak.
Authors: Yuping Lin, Pengfei He, Han Xu, Yue Xing, Makoto Yamada, Hui Liu, Jiliang Tang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: Interactive, Individualized Professional Learning for Elementary School Teachers: Enhancing Content and Pedagogical Content Knowledge as a Basis for Improving Practice, G125128811, G1432373144, NeTS: Small: Exploring the Non-Standardized Polices, Operations, and Requirements for 5G Cellular Networks and Beyond: Advancing the Modeling, Tools, and Evaluation, III:Medium:Computation and Communication Efficient Distributed Learning, Collaborative Research: SaTC: CORE: Medium: Safeguarding Next-Generation Emergency Services (NG-9-1-1) over Cellular Networks: From Design to Practice, Intelligent, Adaptive Program with Just-in-time Feedback for Preservice Teachers, Collaborative Research: III: Medium: Graph Neural Networks for Heterophilous Data: Advancing the Theory, Models, and Applications, TRTech-PGR: Maize Hub: An Open-source Simulation Platform & Resources Integrating Extensive Field Trial Data, DNA Sequences, and Historical Weather Records, G7452299184, TRTech-PGR: Connecting sequences to functions within and between species through computational modeling and experimental studies
OpenAlex ID: W4404783313