Work

A bibliometric review, systematic evidence synthesis, and structured quantitative synthesis of retrieval-augmented and agentic large language models for software repair

Year: 2026
Type: article
Abstract: Large Language Models (LLMs) now generate patches, call tools, retrieve context, and iterate on test feedback, yet published gains are usually reported as counts of patches that pass a test suite on public benchmarks. This review asks whether retrieval-augmented and agentic designs yield more independently correct repairs at comparable cost. Journal articles issued from 2022 to 2026 were retrieved from the OpenAIRE Graph and Crossref with the full Boolean string, supplemented by title screening where abstracts were unavailable. A three-layer design combined a bibliometric map of 151 articles, an abstract-level evidence map and technical appraisal of 67 core empirical studies (11 in full text), and a structured synthesis of matched within-study contrasts. Output grew from 19 articles in 2024 to 83 in 2026, concentrated in five software engineering venues. Coupling analysis separated learning-based repair, agentic repository-level repair, and patch-validity research, with weak modularity indicating a shared intellectual base dominated by Defects4J. Agents, retrieval, and repository-level tasks rose sharply between 2024-2025 and 2026, whereas attention to contamination and patch correctness stayed flat. Only 11% of core abstracts mentioned the plausible versus correct distinction and 11% addressed contamination. Seven studies yielded ten matched contrasts; the nine with magnitudes showed relative gains of 6% to 56%, but none reported paired discordant outcomes or variances, so no pooled estimate was defensible. Memorization and production evidence indicate that benchmark gains overstate deployable correctness. The review proposes a minimum reporting set built on paired per-bug outcomes, budget parity, independent oracles, and contamination audits.
Language: English
Open Access status: hybrid
OpenAlex ID: W7233593676
FWCI:
Cites:
Cited by:
Related to: