Failures Are Rocket Fuel: Verified Per-Model Memory for Local Models in the Unity Editor
Year: 2026
Type: preprint
Abstract: A recent census of 10,400 single-pass Unity generations from four open-weight 7-30B models found that none compiled into a runnable scene, and concluded that the bottleneck is missing engine-specific knowledge. We agree with the diagnosis and disagree that it is a ceiling. With the editor kept in the loop and the missing knowledge supplied as context at run time, three local 7B-14B models complete 95.0-98.3% of 48 Unity 2022.3 editor tasks (10 runs each, every verdict read from the real scene in a headless editor). We then give each model a verified record of its own: only outcomes a verifier judged, with what the model claimed beside what the editor found, recalled before each request and written back only as verified. For live work without a specification, reading the user's request with the same reader that checks the model's claims raises the precision of a "win" from 96.35% to 99.64% on 10,903 recorded completions, losing no real win. With its record, a model passes more of its weakest tasks: pooled over three models and seven tasks, 139 of 210 runs without a record, 181 with its own record and 179 with a record that never held those tasks (CMH p = 2e-07 and 3e-07). The gain is two models' (Morpheus 14B: 40 to 60 and 59 of 60; Gemma 4 12B: 60 to 82 and 89 of 90); for Qwen 7B the record did not help. The transcripts show why: models copy the form of the record's lines. Two cautions came from the same measurements: about 8,000 tokens of off-task chat history cost every model (Morpheus 14B: 77 to 59 of 80, p = 2e-05), and an ungated memory teaches false lessons (1,211 of one model's recorded failures were the harness's fault). Built with Claude (Anthropic) in Claude Code; see the paper's acknowledgements.
Parent work: CuppaTea1983/Leviathan: Leviathan-Setup-v1.0.0.exe
Author: Neal, Michael Ricky
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225988004
