Knowledge Distillation from ResNet-50 to MobileNetV3-Small for Breast Cancer Classification in Ultrasound Images
Year: 2026
Type: article
Abstract: Early detection of breast cancer is essential for reducing mortality, but deploying large deep learning models in resource-constrained clinical settings remains challenging. This study evaluates knowledge distillation (KD) from a ResNet-50 teacher to a MobileNetV3-Small student for classifying breast ultrasound images as benign, malignant, or normal, compared with the same student trained without KD. We partitioned the Breast Ultrasound Images Dataset (BUSI), containing 780 images from 600 patients, using a fixed, stratified 70/15/15 image-level split. Unavailable patient identifiers prevented patient-level splitting, constituting a study limitation. We applied Total Variation Chambolle denoising before training. Grid search selected temperature T = 2 and weighting coefficient α = 0.5, achieving 88.89% validation accuracy. Across five random seeds, the KD student achieved 82.05% ±1.60% test accuracy and 80.93% ±2.04% macro F1-score, compared with 82.74% ±1.64% and 81.88% ±2.17% for the no-KD student, and 83.59% ±4.25% and 82.41% ±5.62% for the teacher. Equivalence testing supported KD–teacher equivalence within a ±5-percentage-point margin (p=0.0498). Paired Wilcoxon tests found no evidence of improvement over the no-KD baseline (p=0.625) or denoising-free training (p=0.4375). The student had 15.5 times fewer parameters and approximately 2.5 times faster inference than the teacher. These findings support compact models for breast ultrasound classification but provide no evidence that KD outperforms simple fine-tuning of the same lightweight architecture on this dataset.
Source: JURNAL INFOTEL
Institutions: Universitas Ma Chung, Universitas Kaltara
Language: English
Topic: AI in cancer detection
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $65
OpenAlex ID: W7225120963
