Evaluating the robustness of explainable AI methods to adversarial attacks in intrusion and malware detection
Year: 2026
Type: article
Abstract: Intrusion detection systems are increasingly paired with eXplainable AI (XAI) so that security analysts can understand and act on machine learning and deep learning model decisions. The explanations themselves, however, may be fragile: a perturbed input can change not only the predicted label but also the feature-importance narrative that justifies it. This paper presents a systematic evaluation of how two gradient-based untargeted evasion attacks, the fast gradient sign method and projected gradient descent (PGD), affect three widely used explainers in intrusion and malware detection: DALEX for dataset-level global importance, SHAP at both global and local granularity, and Integrated Gradients (IG) for local attribution. We quantify explanation stability with three complementary measures, Top- k feature overlap, cosine similarity between attribution vectors, and $$L_1$$ attribution distance, computed over the full test set rather than on individually selected samples. We further separate two effects that are commonly conflated; explanation drift is reported separately for adversarial inputs whose predicted label changes and for those whose predicted label is preserved, which isolates explanation instability from prediction instability. Experiments on CICMalMem22 (obfuscated malware memory forensics) and CICIDS17 (network intrusion) show measurable but method- and dataset-dependent drift when predictions are preserved; the mean Top-5 overlap ranges from 0.586 to 0.892, while cosine similarity ranges from 0.875 to 0.962. The strongest preserved-prediction drift occurs for SHAP on CICMalMem22 under untargeted PGD. These findings indicate that XAI constitutes an additional attack surface in cybersecurity pipelines and that evaluating an IDS should include explanation-robustness measures alongside accuracy and F1-score.
Source: Discover Artificial Intelligence
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,590
OpenAlex ID: W7225406631