Work

Test vs Mutant: Adversarial LLM Agents for Robust Unit Test Generation

Year: 2026

Type: article

Abstract: Software testing is a critical, yet resource-intensive phase of the software development lifecycle. Search-based approaches typically achieve high coverage but produce tests with low readability, whereas large language model (LLM)-based methods generate more human-readable tests but often suffer from low coverage and compilability. While the majority of research efforts have focused on improving test coverage and readability, comparatively less attention has been paid to enhancing the robustness of bug detection. To address this gap, we propose AdverTest, a novel adversarial framework for LLM-powered test case generation that pairs a test case generation agent T with a mutant generation agent M: M persistently creates mutants "hacking" the blind spots of T's current test suite, whileT iteratively refines its tests to "kill" the challenging mutants, with the interaction guided by both coverage and mutation scores. Experimental results on Defects4J show that our approach improves fault detection rates by 8.56% over the best existing LLM-based methods and by 50.20% over EvoSuite, while remaining competitive on line and branch coverage.

Source: Proceedings of the ACM on software engineering.

DOI: 10.1145/3832122

Authors: Pengyu Chang, Yixiong Fang, Silin Chen, Yuling Shi, Beijun Shen, Xiaodong Gu

Institution: Shanghai Jiao Tong University

Language: English

Topic: Software Testing and Debugging Techniques

Subfield: Software

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Natural Science Foundation of Shanghai, National Key Research and Development Program of China

Awards: G4906496431, G5498251246