Risk-Aware Self-Play Reinforcement Learning for Secure and Capable Code-Generating Agents
Year: 2026
Type: preprint
Abstract: We study risk-aware self-play in finite-action code repair, separating functional success from test-integrity and forbidden-call violations. From 591 MBPP/HumanEval programs, 395 mutation tasks yield 4,527 candidates. On 133 external HumanEval tasks, the full policy reduces expected violations from 68.55% to 0.64% versus public-test-only RL, but fixed multiobjective rewards achieve slightly higher joint success, 72.78% versus 72.37%. Additional experiments execute 5,296 alias/data-flow probes and matched controls, investigate selected-action sparse verification, and test 30 functions from Python, JavaScript and TypeScript repositories. Detector blind spots, identical safe/unsafe feature representations and weak repository transfer limit the safety interpretation. Zero greedy violations on 133 tasks yields only an illustrative 2.23% upper risk bound under iid assumptions. These results support verifier-aware rewards without establishing a consistent self-play advantage or a formal deployment guarantee.
Source: Research Square
Authors: Yan-Cheng Hsu, Duo Xu, Ziwei Wang
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225644782