Year: 2026
Type: conference-paper
Abstract: Prompt injection attacks pose a serious security risk to large language models in real-world applications.However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in longcontext settings largely unexplored.This gap leads to a substantial overestimation of the effectiveness of current defenses.In this paper, we bridge the gap by introducing Long-PIBench, a long-context benchmark for prompt injection covering 4 realistic application scenarios: paper peer review, resume screening, code review, and email summary.For each scenario, we construct a synthetic dataset and a real-world dataset, with context lengths ranging from thousands to tens of thousands of tokens.The evaluation results on LongPIBench reveal significant vulnerabilities of prompt injection defenses under long-context settings: even simple heuristic prompt injection attacks achieve high success rates and frequently bypass state-of-the-art defenses.We hope Long-PIBench can serve as a practical benchmark for systematically evaluating prompt injection defenses in realistic long-context scenarios.Our code and data are available at: https: //github.com/liu00222/LongPIBench. Four Test Scenarios Eight Datasets Eight Large Language Models Eight Attacks Nine Detection Methods Six Prevention Methods Paper review Resume screening Code review Email summary Synthetic datasets Real-world datasets Llama-3.1-8B-InstructApertus-8B-Instruct •••••• GPT-4.1 Naïve attack Escape character •••••• Authority spoof Attention tracker DataSentinel •••••• Prompt guard 2 Instructional prevention Delimiters •••••• MetaSecAlign
Institutions: Pennsylvania State University, Duke University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Funder: National Science Foundation
Awards: Collaborative Research: SaTC: CORE: Medium: Securing LLMs against Prompt Injection Attacks, CICI: UCSS: Secure Machine Learning as a Service for Collaborative Scientific Research, Collaborative Research: SaTC: CORE: Small: Securing Recommender Systems against Data Poisoning Attacks, Collaborative Research: SaTC: EDU: A Lab-based Curriculum for Watermarking AI-Generated Content: Theory, Algorithms, and Robustness Testing, Collaborative Research: SaTC: CORE: Medium: Towards Secure Federated Learning, AI Institute for Edge Computing Leveraging Next Generation Networks (Athena), Collaborative Research: SaTC: CORE: Medium: Securing LLMs against Prompt Injection Attacks
OpenAlex ID: W7204759488