Improving Validity of LLM-Synthesised Test Suits for Compiler Fuzzing
Year: 2025
Type: dissertation
Abstract: Modern compilers represent complex software systems where traditional testingapproaches struggle to adapt to evolving software semantics and programmingparadigms. The emergence of Large Language Models offered potential for diversetest generation without requiring software rewrites for different languages andframeworks, but their practical application has been limited by low validity rates,wasting computational resources and restricting testing depth.In this thesis, I present a feedback-driven repair framework that addresses validityproblems in LLM-based compiler testing by combining automated error correctionwith coverage-guided programme transformation. My approach constructs a re-pair pipeline that utilises compiler diagnostics and sanitizer outputs to guide localLLMs in correcting static and dynamic errors, transforming invalid programmesinto high-quality seeds. The framework incorporates a semantic change scoreformulation that quantifies programme transformations and enables systematicguidance towards coverage objectives. However, balancing repair effectivenesswith semantic preservation while maintaining programme interestingness presentsa significant challenge.I describe novel coverage-guided automated programme repair (CG-APR) tech-niques where invalid programmes are automatically transformed using structuredprompting and iterative refinement to target untested compiler components. Therepair process uses semantic change analysis based on programme structure andbehavioural differences.I evaluated ReFuzzer on C++ programmes from black-box, grey-box, and white-box fuzzers targeting LLVM/Clang. The results demonstrate validity rates of 96–97% (improving from a 47–49% baseline) and significant coverage improvementsin optimisation passes, with absolute gains of up to +21.2% in inlining and +17.0%in dead code elimination.
Author: Shree, Iti
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7083681669
Cites:
Cited by:
Related to: