Year: 2026
Type: conference-paper
Abstract: Large Language Models (LLMs) are increasingly deployed via third-party system prompts downloaded from public marketplaces.We identify a critical supply-chain vulnerability: conditional system prompt poisoning, where an adversary injects a "sleeper agent" into a benign-looking prompt.Unlike traditional jailbreaks that aim for broad refusal-breaking, our proposed framework, PARASITE, optimizes system prompts to trigger LLMs to output targeted, compromised responses only for specific queries (e.g., "Who should I vote for the US President?") while maintaining high utility on benign inputs.Operating in a strict blackbox setting without model weight access, PAR-ASITE utilizes a two-stage optimization including a global semantic search followed by a greedy lexical refinement.Tested on opensource models and commercial APIs (GPT-4omini, GPT-3.5),PARASITE achieves up to 70% F1 reduction on targeted queries with minimal degradation to general capabilities.We further demonstrate that these poisoned prompts evade standard defenses, including perplexity filters and typo-correction, by exploiting the natural noise found in real-world system prompts.Our code and data are available at https://github.com/vietph34/ PARASITE.WARNING: Our paper contains examples that might be sensitive to the readers!* This work was conducted prior to joining IU Dual Behavior Demonstration 1. Attacker Crafts a Poisoned PromptA helpful tool for learning about historical events and figures.
Authors: Viet Pham, Thai Le
Institution: Indiana University Bloomington
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G1453181110, Track 4: Advanced CI Coordination Ecosystem: Monitoring and Measurement Services, G3251540411, G3562386289, Track 1: ACCESS Resource Allocations Marketplace and Platform Services (RAMPS), G4010157413, ACO: An Open CI Ecosystem to Advance Scientific Discovery (OpenCI), Track 3: COre National Ecosystem for CyberinfrasTructure (CONECT), G5367127336, G5464919783, Track 2: Customized Multi-tier Assistance, Training, and Computational Help (MATCH) for End User ACCESS to CI
OpenAlex ID: W7166869285