Year: 2025
Type: preprint
Abstract: Software vulnerabilities such as buffer overflows and SQL injections are a major source of security breaches. Traditional methods for vulnerability detection remain essential but are limited by high false positive rates, scalability issues, and reliance on manual effort. These constraints have driven interest in AI-based approaches to automated vulnerability detection and secure code generation. While Large Language Models (LLMs) have opened new avenues for classification tasks, their complexity and opacity pose challenges for interpretability and deployment. Sparse Autoencoder offer a promising solution to this problem. We explore whether SAEs can serve as a lightweight, interpretable alternative for bug detection in Java functions. We evaluate the effectiveness of SAEs when applied to representations from GPT-2 Small and Gemma 2B, examining their capacity to highlight buggy behaviour without fine-tuning the underlying LLMs. We found that SAE-derived features enable bug detection with an F1 score of up to 89%, consistently outperforming fine-tuned transformer encoder baselines. Our work provides the first empirical evidence that SAEs can be used to detect software bugs directly from the internal representations of pretrained LLMs, without any fine-tuning or task-specific supervision. Code available at https://github.com/rufimelo99/SAE-Java-Bug-Detection
Source: arXiv (Cornell University)
Authors: Melo, Rui, Mamede, Claudia, Catarino, Andre, Abreu, Rui, Cardoso, Henrique Lopes
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: Laboratório de Inteligência Artificial e Ciência de Computadores, G3066072003, G323769159, G4387801833, G4746484675, Synthetic and scalable data platform for medical empowered AI, G5485649731, G670063171, G8241006718, G8453797156, Laboratório de Inteligência Artificial e Ciência de Computadores, G966035586
OpenAlex ID: W4415200197