Knowledge Distillation from ResNet to MobileNet for Accurate On-Device Face Recognition
Year: 2025
Type: article
Abstract: The development of efficient facial recognition systems on low-resource devices requires models to optimize computational cost and performance for discrimination tasks. With this consideration, we introduce Adaptive Feature-Logit Distillation (AFL-KD), a novel framework that combines logit imitation, mid-level feature alignment, and weighted learning of losses with a specific focus on open-set face verification. During each train step, AFL-KD continuously adjusts the relative weights of the cross-entropy, logit distillation, and feature alignment components by matching the magnitudes of their gradients, thereby alleviating an upper bound on the angular-margin error of the student model. In the process of compressing a 283 MB ResNet teacher into a 7.2 MB MobileNet, AFL-KD achieves a negligible drop of 1.5 points in verification accuracy.Empirical evaluations on the CASIAWebFace dataset show that distilled MobileNet achieves 95.7% accuracy, 77.0% recall, and an F1 score of 86. 9%, closely approaching the ResNet teacher (97.2% accuracy, 84.8% recall) while shrinking the model size from 283.3 MB to 7.2 MB—a 39× compression. Compared with a conventionally trained MobileNet baseline (87.3% accuracy, 62.8% F1 at 14 MB), the distilled model delivers +8.4% absolute accuracy and +24.1% F1 improvements while halving the memory footprint. These results confirm that knowledge distillation can yield highly accurate yet resource-efficient face recognition suitable for mobile and embedded applications.
Authors: İBRAHİMOĞU, Nadir, Aytekin, Mehmet Cem, Yıldız, Furkan
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6949784505