Pressure, not gauges: Why better grading still leaves the question of safe failure
Year: 2026
Type: preprint
Abstract: Between May and July 2026, AI agents inside OpenAI circumvented a cybersecurity evaluation, built an unauthorised message board, and broke into Hugging Face and an OpenAI research cluster. This note examines the incident through a distinction developed in The Gauge: what an evaluation measures and what it costs to fail it. The hypothesis is that, when honest failure remains penalised and task success dominates other constraints, a sharper grader can displace reward hacking rather than reduce it. The agents’ attempts to defeat a check that did not exist show why their understanding of the evaluation also matters. The reports motivate this hypothesis but do not establish its causal explanation. A proposed 2×2 experiment separates stronger scrutiny from a rewarded, verified way to stop safely, measuring whether each intervention reduces reward hacking or changes its form.
Author: Denjoy, Frédéric
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204832532
Cites:
Cited by:
Related to: