Work

Leveraging Large Language Models for Scalable and Explainable Cybersecurity Log Analysis

Year: 2025

Type: article

Abstract: The increasing complexity and volume of cybersecurity logs demand advanced analytical techniques capable of accurate threat detection and explainability. This paper investigates the application of Large Language Models (LLMs), specifically qwen2.5:7b, gemma3:4b, llama3.2:3b, qwen3:8b and qwen2.5:32b to cybersecurity log classification, demonstrating their superior performance compared to traditional machine learning models such as XGBoost, Random Forest, and LightGBM. We present a comprehensive evaluation pipeline that integrates domain-specific prompt engineering, robust parsing of free-text LLM outputs, and uncertainty quantification to enable scalable, automated benchmarking. Our experiments on a vulnerability detection task show that the LLM achieves an F1-score of 0.928 ([0.913, 0.942] 95% CI), significantly outperforming XGBoost (0.555 [0.520, 0.590]) and LightGBM (0.432 [0.380, 0.484]). In addition to superior predictive performance, the LLM generates structured, domain-relevant explanations aligned with classical interpretability methods. These findings highlight the potential of LLMs as interpretable, adaptive tools for operational cybersecurity, making advanced threat detection feasible for SMEs and paving the way for their deployment in dynamic threat environments.

Source: Journal of Cybersecurity and Privacy

DOI: 10.3390/jcp5030055

Authors: Giulia Palma, Gaia Cecchi, Mario Caronna, Antonio Rizzo

Institution: University of Siena

Language: English

Topic: Network Security and Intrusion Detection

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,207

Funder: European Regional Development Fund