Work

FedHDE: an efficient federated learning model based on hyper-knowledge distillation

Year: 2026

Type: article

Abstract: The heterogeneity of data between different clients limits the performance of federated learning training, and hyper-knowledge distillation has emerged as a potential solution to address heterogeneous data. However, existing hyper-knowledge distillation methods often sacrifice efficiency to improve model performance. To address these challenges, this paper proposes FedHDE, a novel federated learning framework based on hyper-knowledge distillation and efficient tensor optimization. FedHDE integrates a Dynamic Client Selection strategy within a teacher–assistant–student hierarchical structure to enhance global aggregation accuracy and reliability without relying on public datasets or server-side models. Furthermore, an efficient tensor optimization algorithm is introduced to improve numerical stability and reduce computation time. Theoretical analysis demonstrates that FedHDE achieves a convergence rate of $$O(1/\sqrt{T})$$ O ( 1 / T ) , ensuring efficient and stable learning. Experimental results on the CIFAR10 dataset show that FedHDE improves both local and global accuracy by 0.14%–4.32%, while reducing training time by 65.28% compared to state-of-the-art baselines.Code is public at https://anonymous.4open.science/r/FedHDE .

Source: Journal of King Saud University - Computer and Information Sciences

DOI: 10.1007/s44443-026-00503-1

Authors: Xingang Zhang, Chuang Liu, Xiaofan Shao, Tingzhi Li, He Li, Dongyan Zhang

Institutions: Nanyang Normal University, Henan Normal University

Language: English

Topic: Privacy-Preserving Technologies in Data

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,990

Funder: Natural Science Foundation of Henan Province