Artificial Intelligence in Software Testing: A Systematic Survey of Test Generation, Mutation Testing, and Flaky Test Management
Year: 2026
Type: preprint
Study design: Systematic Review
Abstract: Software testing consumes a large share of development effort, and its most labor-intensive tasks — writing test cases, judging whether outputs are correct, and keeping test suites reliable — are precisely the tasks where artificial intelligence has recently made the largest advances. This paper presents a systematic survey of artificial intelligence in software testing. It traces the field from classical and search-based test generation, through symbolic execution and fuzzing, to the present generation of large language model methods; it reviews how AI is used for automated test generation, assertion and oracle construction, mutation testing, flaky-test detection and management, and test prioritization and fault prediction; it examines the evidence for each family of techniques, including where empirical results are strong, where they are mixed, and where they rest on narrow benchmarks; and it analyzes the central challenges of the field — the oracle problem, evaluation validity, non-determinism, cost, and the risk of over-trusting generated tests. The survey concludes that AI has moved testing from a largely manual craft toward a semi-automated discipline, but that generated tests still require human review, measured adequacy criteria, and disciplined suite maintenance, and that the field's most valuable direction is the combination of learned models with classical, coverage-guided, and mutation-based methods rather than the replacement of one by the other.
Author: Mahmoudpour, Aref
Institution: Islamic Azad University, Tehran
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225557761