FedAK: a semi-supervised one-shot framework for heterogeneous federated learning via feature-level attention-based knowledge distillation
Year: 2026
Type: article
Abstract: Federated learning (FL) enables multiple devices to collaboratively train a shared model while keeping their data localized, thereby preserving privacy. Despite its promise, FL continues to face key challenges such as model heterogeneity, high communication overhead, and performance degradation under non-independent and identically distributed (non-IID) data. This paper introduces FedAK, a semi-supervised one-shot FL framework that integrates feature-level attention with knowledge distillation to support efficient global learning while reducing data exposure. In FedAK, each client independently trains its local model in a fully supervised manner on private labeled data and transmits only the feature representations of a shared public dataset, greatly reducing communication costs. On the server side, a semi-supervised aggregation strategy is adopted, where an attention-based aggregation module is trained on a small labeled subset to generate informative ensemble features and soft pseudo-labels for a larger unlabeled subset. These pseudo-labels guide a global student model through knowledge distillation, eliminating the need for direct access to client data or logits. Extensive experiments across four benchmark datasets demonstrate that FedAK consistently outperforms four state-of-the-art one-shot FL methods, confirming its effectiveness under heterogeneous and non-IID settings.
Source: Scientific Reports
Institutions: Centre National de la Recherche Scientifique, Sorbonne Université, Université Paris Nanterre, LIP6, Université Paris Sciences et Lettres, Lamsade, Université Paris Dauphine-PSL, Modern University for Business and Science, American University of the Middle East, Holy Spirit University of Kaslik
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7162197948
FWCI:
Cites:
Cited by:
Related to: