Year: 2025
Type: preprint
Abstract: Code auditing is the process of reviewing code with the aim of identifying bugs. Large Language Models (LLMs) have demonstrated promising capabilities for this task without requiring compilation, while also supporting user-friendly customization. However, auditing a code repository with LLMs poses significant challenges: limited context windows and hallucinations can degrade the quality of bug reports, and analyzing large-scale repositories incurs substantial time and token costs, hindering efficiency and scalability. This work introduces an LLM-based agent, RepoAudit, designed to perform autonomous repository-level code auditing. Equipped with agent memory, RepoAudit explores the codebase on demand by analyzing data-flow facts along feasible program paths within individual functions. It further incorporates a validator module to mitigate hallucinations by verifying data-flow facts and checking the satisfiability of path conditions associated with potential bugs, thereby reducing false positives. RepoAudit detects 40 true bugs across 15 real-world benchmark projects with a precision of 78.43%, requiring on average only 0.44 hours and $2.54 per project. Also, it detects 185 new bugs in high-profile projects, among which 174 have been confirmed or fixed. We have open-sourced RepoAudit at https://github.com/PurCL/RepoAudit.
Source: arXiv (Cornell University)
Authors: Guo, Jinyao, Wang, Chengpeng, Xu, Xiangzhe, Su, Zian, Zhang, Xiangyu
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: SHF: Small: AI Model Debugging by Analyzing Model Internals with Python Program Analysis, Proto-OKN Theme 1: Knowledge Graph Construction for Resilient, Trustworthy, and Secure Software Supply Chains, SLES: Whitebox Testing, Debugging, and Repairing for Multi-module Autonomous Vehicles in Near-Collision Traffic Scenarios, SHF: Medium: Principled Co-Reasoning of Software and Natural-Language Artifacts, G8876996369
OpenAlex ID: W4407012199