Prove It First: Reproduction-Gated LLM Repair of Production Exceptions in Laravel Applications
Year: 2026
Type: preprint
Abstract: Large language models (LLMs) can now propose fixes for many bugs, but a fix that merely makes an error disappear is not necessarily correct, and the evidence that a fix works is usually left to the reviewer. We present Lazarus, an open-source self-healing tool for Laravel (PHP) applications that turns an exception captured in production into a reviewable fix only when it can prove three things: a generated test reproduces the failure, a minimal patch turns that test from red to green, and the project's existing test suite still passes. Nothing is published otherwise. We evaluate Lazarus on LaraBugs-29, a benchmark of 20 synthetic and 9 real exceptions; the real ones were mined from the logs of 28 Laravel projects (381 distinct exceptions, 6,481 occurrences). Each fix is additionally judged by a hidden oracle that the tool never sees. With two Claude models, Lazarus produced a verified fix for all 29 bugs, and 27 of 29 fixes (93%) also passed the hidden oracle. The two oracle failures are instructive: in one case the model "fixed" the bad input at the call site instead of the function, and in the other it added a fallback that still crashes when the fallback is missing. When the failing input was instead hidden from the code, as production data usually is, the fixes from the stronger model were correct for all 29 bugs and those from the smaller model for 28: without the input in view, the models repaired the functions rather than the data. The results suggest that reproduction-gated repair is practical for everyday web-application exceptions at a cost of about 13k tokens and half a minute per bug, while also showing that a passing reproduction test is not a proof of correctness.
Author: ALASHQAR, MOHAMMED A. A.
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225823229
Cites:
Cited by:
Related to: