Entropy-driven adaptive knowledge distillation with reliable candidate fusion for imbalanced binary classification
Year: 2026
Type: preprint
Abstract: Class imbalance can impair minority-class recognition and probability reliability, while conventional knowledge distillation often relies on fixed knowledge-transfer settings and a fixed prediction form. In this study, we propose entropy-driven adaptive knowledge distillation with reliable candidate fusion (EAKD-RCF) for imbalanced binary classification. EAKD-RCF consists of two main stages. First, adaptive distillation separates predictive uncertainty from teacher reliability: uncertainty determines the sample-specific distillation temperature, whereas training-fold-level reliability regulates the overall strength of knowledge transfer. Second, student, teacher, and fused predictions are treated as complementary candidates, and the final prediction form is selected using internal validation by jointly considering imbalanced classification performance and probability quality. The proposed method was evaluated on nine public datasets using repeated stratified five-fold cross-validation. EAKD-RCF achieved mean MCC, F1-score, and G-Mean values of 61.96%, 72.55%, and 79.07%, respectively, while maintaining competitive ECE and Brier scores of 7.97% and 10.30%. Stage-wise and ablation analyses further showed complementary contributions from adaptive distillation and candidate selection. These results demonstrate strong average classification performance across heterogeneous datasets while maintaining competitive probability reliability.
Source: Research Square
Institution: Yulin University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7226003031
