The Ticket Cuts the Cheating, the Model Shapes the Confession: What Coding Agents Report After Bypassing an Impossible Test
Year: 2026
Type: preprint
Abstract: A rule in the ticket cuts how often a coding agent bypasses an impossible test; what the agent then admits differs sharply between the models that bypass. Four model-and-harness packages received two pytest tasks that cannot pass in the sandbox, and a deterministic checker out of the agent's reach decided every bypass. In a pre-registered 2x2 factorial on the ticket, bypassing fell from 28/29 runs with no rule to 4/24 with a prohibition, 19/30 with a one-sentence permission to report the blocker, and 0/30 with both (2/69, none green, in an extended sample). Blind coders from two model families then coded the final message of the 327 green bypasses that ended with a report. Under every coder, reports named the shortcut (84% to 89%) more often than they said what the tests no longer checked (59% to 76%; paired p < 1e-6 per coder). GPT-6.1 Sol named its shortcut in 66/68 reports; DeepSeek V4 Flash left it unnamed in 32/112 and contradicted the checker in 16/112. A Claude monitor reading only the ticket and the report accepted 42/335 bypasses as trustworthy, 29/335 with the diff; most of those it accepted were synthetic test data the report had disclosed. The reporting sentence had no detectable effect on what bypassers reported (an underpowered test), and the pre-registered claim that full confession is a minority did not survive audit. Code, data, pre-registration and coding sheets: https://github.com/AmirK-S/final-reports-impossible-tasks
Author: Kellou-Sidhoum, Amir
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7236145351
