Contained or Escaped? Measuring Sandbox Escape and Auto-Reviewer Decisions in AI Coding Agents
Year: 2026
Type: preprint
Abstract: Artificial intelligence (AI) coding agents now run shell commands in a default sandbox and must request permission, often screened by a large language model (LLM) auto-reviewer, before a command may use the network or write outside the workspace. Most prompt-injection benchmarks for coding agents assume unrestricted execution. We built SARD-Bench, 240 poisoned-workspace cases in four families and three stealth tiers, with a harness whose single tool makes the agent declare and justify the permissions it needs. We ran Claude Sonnet 5.5, GPT-5.1 and Llama 4 Maverick once per case (720 trajectories), executed the two-turn family in a macOS Seatbelt sandbox, and had a GPT-4o-mini reviewer judge every harmful command with and without the agent’s rationale. All labels were audited by hand in two passes. The two proprietary models ran the payload inside the sandbox in 28.3% of runs and requested the privileges to escape in 6.5%. Llama 4 Maverick escaped in 44.6%. Stealthier files raised in-sandbox execution from 10.6% to 50.6% but left escape rates unchanged. With the agent’s rationale, the reviewer blocked 126 of 138 escape requests, against 64 from the command alone. Claude Sonnet 5.5 returned provider refusals in 140 runs, and an indicator-matching scorer missed half of all escapes. Evaluations of sandboxed agents should report where a payload would run and audit their labels by hand.
Source: Preprints.org
Authors: Deep Gandhi, Young Lee
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7234928151
