The Measurement Framework for Underestimated Failure Modes of Past-only Query Policies in Open-World Malware Family Classification
Year: 2026
Type: article
Abstract: Laboratories that evaluate open-world malware family classifiers typically ship a query or abstention policy chosen on a historical proxy window. The work argues that this past-only selection step constitutes an underestimated failure mode: the proxy-optimal policy can be unsafe on the next unseen-family window even when the candidate grid is fixed. This problem differs from training a better classifier, per-sample rejection, and learning a detection-and-labeling policy with future adaptation. It presents a rigorous measurement framework: a frozen past-only protocol, a ledger of negative selectors, split safety and efficacy estimands, and one registered instance named MPSR-RSG that never reads current future labels. On development windows covering BODMAS PE and EMBER2024 with 18 comparisons, the instance exhibits zero failures. Across 12 baseline-failure windows, its hierarchical-bootstrap 95% CI for failure-rate reduction is [0.25, 1.00].For the preregistered held-out efficacy population consisting of Avast-CTU CAPEv2 and MOTIF with 9 baseline-failure windows, the corresponding CI is [0.22, 1.00].BODMAS Phase 0a cutoffs are retained within the safety monitor rather than the confirmatory efficacy population. This arrangement follows considerations of source role, chain integrity, and estimand identifiability, rather than an attempt to improve the headline interval by excluding these data. The core claim supported by experiments is that past-only selection requires high-priority evaluation under these constraints, and the work does not prove the existence of a universal selector.
DOI: 10.54097/evhnkb11
Authors: Jiali Che, Haoqing Ren, Tingting Sun
Institution: Sias University
Language: English
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7224764788