Execution-Feedback Code Repair Agents with Test-Guided Iterative Refinement
Year: 2025
Type: article
Abstract: Code agents often produce near-correct programs that fail executable checks. This study evaluates an execution-feedback repair agent (EFR) that converts test outcomes into structured evidence, localizes suspicious code, proposes bounded abstract-syntax-tree edits, and re-ranks remaining patches after failed attempts. All 164 HumanEval tasks were used. After canonical solutions passed the official checks, one reversible fault was introduced per task across six edit families. The 1,582 dynamically executed assertions were split into 1,260 feedback and 322 held-out assertions, and 1,470 localized proposals were executed. A task-grouped five-fold logistic ranker supplied a patch prior; coverage, exception compatibility, and test-score changes supplied runtime evidence. EFR repaired 83.54% of programs within five attempts and 87.80% within ten (95% Wilson interval, 81.91%-91.97%), compared with 82.32% for learned fixed-order search, 66.46% for coverage search, and 57.32% for random search. The ten-attempt policy averaged 2.77 attempts and 914.6 lexical tokens. Only one feedback-passing patch failed held-out verification. The results show that a learned prior improves early efficiency, while bounded execution and evidence updates convert local failures into verified repairs.
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7220154550