Get started
Work

U-XAI-IDS: A Unified Benchmark for Domain-Feasible and Adversarially-Robust Explainability in Intrusion Detection

Year: 2026
Type: article
Abstract: Post-hoc explainers such as SHAP, LIME, and Integrated Gradients (IG) are increasingly deployed to justify the decisions of machine-learning intrusion detection systems (IDS), yet the field lacks a standardized way to compare them: existing studies use different models, datasets, and single-axis metrics, and almost none check whether the perturbations an explainer generates are even valid network records. I present U-XAI-IDS, a unified, reproducible benchmark that evaluates explanations for network IDS along four axes—fidelity (deletion/insertion AUC), stability (Lipschitz continuity), domain feasibility, and inter-explainer agreement—together with an adversarial-robustness axis, across four detector families (Random Forest, LSTM, and two Graph Neural Network variants) and two datasets (NSL-KDD, CICIDS2017), all with multi-seed 95% confidence intervals. The central contribution is Domain-Constraint-Aware (DCA) perturbation and the associated Domain Feasibility Rate (DFR) metric: standard covariance-based LIME neighbourhoods on NSL-KDD violate one-hot protocol/service/flag constraints in 100% of samples (DFR = 0.00), whereas DCA repairs them to full validity (DFR = 1.00) while improving stability. I further find that (i) IG is the most stable explainer for deep and graph detectors (0.998±0.000) and the only method whose feature rankings ever agree with SHAP, (ii) LIME and SHAP disagree systematically (Spearman ρ < 0 in 39 of 40 model/dataset configurations, the sole exception statistically indistinguishable from zero), and (iii) high stability does not imply adversarial robustness. U-XAI-IDS and all code are released to support reproducible, apples-to-apples comparison of IDS explanations.
DOI: 10.5281/zenodo.22290978
Language: English
Open Access status: green
OpenAlex ID: W7208736023
FWCI:
Cites:
Cited by:
Related to: