When Does Temporal Adaptation Help in Static-Feature Malware Classification? A 27-Method Benchmark
Year: 2026
Type: preprint
Abstract: Machine learning models for malware family classification degrade over time as malware evolves—a phenomenon known as concept drift. We present the most comprehensive empirical benchmark of temporal adaptation for malware classification to date: 27 method configurations spanning continual learning (EWC, SI, LwF, A-GEM, GDumb, DER++, ER-ACE), prompt-based learning (L2P), progressive neural networks, domain adaptation (DANN), replay methods, frozen-backbone ensembles, last-layer fine-tuning, XGBoost ensembles, cumulative retraining baselines, and prior probability recalibration—evaluated using pre-computed EMBER static features (2,381 dimensions) across 10 months of drift on BODMAS (57K samples, 12 categories) with 5 random seeds, cross-validated on EMBER 2018 (12 of 27 methods, 175K samples, 10 families), and analyzed with GEE models, bootstrap confidence intervals, and bias-variance decomposition. Our central finding is that under domain-incremental drift with pre-computed static features, full-model neural adaptation consistently underperforms frozen-backbone strategies: an MLP fine-tuned monthly drops 5.5 percentage points below a static MLP, and all standard continual learning methods cluster around this adapted baseline (72.8–79.5%), none reaching the static MLP. Diagnostic experiments reveal why: (1) last-layer-only fine-tuning matches the best neural method (L2P, 90.8%) without any ensemble, proving that backbone overparameterization—not adaptation itself—causes failure; (2) prior probability recalibration captures only +2.3pp, confirming drift is primarily covariate shift; (3) a majority-class collapse affects 9 adapted methods on the same month, indicating shared failure modes. Methods that avoid full-model neural adaptation achieve the highest mean accuracy: CumulativeXGBoost (91.5%), XGBoost Ensemble (91.3%), L2P (90.8%), LastLayer-FineTune (90.8%), LastLayer-Fresh (90.2%), and FBE (89.8%), while CumulativeRetrain (86.0% ± 4.6pp) improves over static baselines but still falls 5.5pp below CumulativeXGBoost, confirming the neural overparameterization penalty even with maximal training data. The frozen-backbone principle is confirmed across both datasets, binary detection, architecture ablation (128/256/512 hidden dims), and all feature subsets. A class-incremental evaluation reveals that softmax confidence is anti-correlated with novelty for functionally overlapping families, requiring dedicated out-of-distribution detectors in production. The 27 implementations, evaluation scripts, and analysis code are available from the author upon reasonable request as a community benchmark resource.
Author: Bhimavarapu, Satwik
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225653609
Cites:
Cited by:
Related to: