Unified SAST benchmark: Compare AI-driven and traditional static analyzers
Year: 2025
Type: software-paper
Abstract: In recent years, the growing need for robust security in software development has driven the adoption of static source code analysis tools to detect vulnerabilities early in the software lifecycle. While traditional static analyzers have been widely used, the success of AI in other domains suggests that AI-driven approaches can enhance detection accuracy and reduce false positives. However, a standardized benchmark for comparing AI-based and conventional static analysis tools remains lacking. In this paper, we present an enhanced version of an existing benchmark, extended to support comprehensive comparisons between AI-powered and traditional static code analyzers. The current version includes a dataset of real-world JavaScript vulnerabilities, structured as vulnerable and fixed code pairs—an approach shown to be critical for realistic model evaluation. While the benchmark currently focuses on JavaScript, it is designed to be extensible to other languages in future work. We also introduce a sample static analyzer powered by an AI model to demonstrate practical usage and show how other researchers can easily integrate their own models. Overall, our contributions aim to support reproducible, meaningful evaluation and foster progress toward more secure software development practices.
Source: SoftwareX
Authors: Tamás Viszkok, Péter Hegedűs
Institution: University of Szeged
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,200
OpenAlex ID: W4414043122