Scaling Laws Do Not Predict Adversarial Robustness in LLM-Based Security Detection Systems
Year: 2026
Type: preprint
Abstract: Large language models (LLMs) are increasingly deployed as automated defenders in security operations, generating detection rules from attack telemetry. A common assumption is that larger models produce more robust detections. We challenge this assumption through DUEL (Dual Unified Evasion Loop), an adversarial red-teaming framework in which an LLM Attacker generates synthetic Microsoft Sentinel-compatible telemetry and an LLM Defender generates KQL detection rules across 38 MITRE ATT&CK techniques. We introduce DABS (Dual Adversarial Benchmark Score), a standardized 0-100 metric measuring defender robustness across five weighted components. Benchmarking five open-weight models from 3.8B to 14B parameters, we find a power law DABS = 65.36 x P^-0.087 with R² = 0.055, indicating that model size explains less than 6% of the variance in adversarial detection robustness. mistral:7b achieves the highest DABS score (66.27) despite being half the size of qwen2.5:14b (55.97). The overall attacker evasion rate across all 38 techniques is 58.4%, with Discovery and Impact tactics reaching 91.3% and 87.0% respectively. Framework, benchmark, and dataset available at https://github.com/0xDanielSec/duel-framework
Author: Gomes Viana Rocha, Daniel
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7160711121
Cites:
Cited by:
Related to: