Year: 2023
Type: preprint
Abstract: Large language models (LLMs) are vulnerable to adversarial attacks that add malicious tokens to an input prompt to bypass the safety guardrails of an LLM and cause it to produce harmful content. In this work, we introduce erase-and-check, the first framework for defending against adversarial prompts with certifiable safety guarantees. Given a prompt, our procedure erases tokens individually and inspects the resulting subsequences using a safety filter. Our safety certificate guarantees that harmful prompts are not mislabeled as safe due to an adversarial attack up to a certain size. We implement the safety filter in two ways, using Llama 2 and DistilBERT, and compare the performance of erase-and-check for the two cases. We defend against three attack modes: i) adversarial suffix, where an adversarial sequence is appended at the end of a harmful prompt; ii) adversarial insertion, where the adversarial sequence is inserted anywhere in the middle of the prompt; and iii) adversarial infusion, where adversarial tokens are inserted at arbitrary positions in the prompt, not necessarily as a contiguous block. Our experimental results demonstrate that this procedure can obtain strong certified safety guarantees on harmful prompts while maintaining good empirical performance on safe prompts. Additionally, we propose three efficient empirical defenses: i) RandEC, a randomized subsampling version of erase-and-check; ii) GreedyEC, which greedily erases tokens that maximize the softmax score of the harmful class; and iii) GradEC, which uses gradient information to optimize tokens to erase. We demonstrate their effectiveness against adversarial prompts generated by the Greedy Coordinate Gradient (GCG) attack algorithm. The code for our experiments is available at https://github.com/aounon/certified-llm-safety.
Source: arXiv (Cornell University)
Authors: Kumar, Aounon, Agarwal, Chirag, Srinivas, Suraj, Li, Aaron Jiaxun, Feizi, Soheil, Lakkaraju, Himabindu
Institutions: Harvard University, University of Maryland, College Park
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1215885793, Collaborative Research: RI: Small: Post hoc Explanations in the Wild: Exposing Vulnerabilities and Ensuring Robustness, Career: Towards a Systematic Characterization of Model Explanations for High-Stakes Decision Making, EFFECTIVENESS AND SCALABILITY OF ROBUSTNESS CERTIFICATIONS AGAINST ADVERSARIAL ATTACKS, CAREER: Information-Theoretic and Statistical Foundations of Generative Models, G4720003262, FAI: Towards Adaptive and Interactive Post Hoc Explanations, G6671297155, Collaborative Research: CIF: Medium: Understanding Robustness via Parsimonious Structures., G8876996369
OpenAlex ID: W4386555960