Comparative Evaluation of Explanation Reliability in Shipment-Level Supply Chain Risk Classification: A Leakage-Checked Benchmark of XGBoost and a Hybrid XGBoost–Neural Framework with SHAP and LIME
Year: 2026
Type: article
Abstract: Machine learning can forecast supply chain disruptions, but black-box models limit transparency, and the reliability of post-hoc explanations is rarely evaluated. We provide a leakage-checked, reproducible comparative benchmark for shipment-level risk classification on a public synthetic dataset of 5000 shipments, comparing XGBoost alone with a hybrid XGBoost–neural model (XAI-SC-RiskNet), both explained with SHAP and LIME. The hybrid (71.60% accuracy, ROC-AUC 0.819) is statistically indistinguishable from XGBoost (72.60%, 0.819; McNemar’s test p=0.123), as repeated cross-validation and a chronological hold-out confirm. On 500 test shipments, the Local Explanation Consensus Index (LECI; top-6 overlap of local SHAP and LIME features) and LIME stability are lower for the hybrid (0.57 versus 0.68; 0.80 versus 0.88), so the hypothesis that hybridization stabilizes explanations is not supported. LECI was modestly associated with misclassification (AUC 0.61) but added no significant information to predictive uncertainty (AUC 0.77); it is therefore a descriptive measure of explanation agreement, which does not establish explanation correctness or better decisions, and not a validated quality-control mechanism. These findings are specific to one synthetic benchmark with a concentrated weather signal and do not show that hybrid architectures are generally inferior.
Source: Information
DOI: 10.3390/info17111181
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,930
OpenAlex ID: W7236827396
