FedKD-MR: Enabling Heterogeneous Federated Learning through Multi-Round Knowledge Distillation
Year: 2026
Type: article
Abstract: In heterogeneous IoT deployments, federated learning is challenged not only by distributed, non-IID data but also by the possibility that participating devices may rely on incompatible model architectures. Under these conditions, classical parameter aggregation becomes difficult to apply because client weights cannot be meaningfully combined. FedKD-MR addresses this problem by shifting coordination from the parameter space to the output space: clients exchange logits computed on a shared, public reference dataset, enabling collaboration across heterogeneous models under a common input-label interface. Its main contribution is to organize this output-space exchange into a multi-round process in which consensus logits are repeatedly aggregated, redistributed, and reused during local distillation and supervised fine-tuning. We evaluate FedKD-MR on CIFAR-10 and CIFAR-100 under heterogeneous, non-IID client configurations, including cases in which parameter aggregation cannot be applied to the full client population. The results show gains over isolated local training. We further evaluate the method on UCI-HAR to examine a sensor-oriented setting and the role of the public anchor dataset. This analysis shows that public-set size, class coverage, and distributional quality directly affect the usefulness of consensus logits. Overall, the results support the use of multi-round logit reuse as a practical alternative for coordinating heterogeneous IoT clients when shared model weights are not feasible.
DOI: 10.1145/3859281
Authors: Claudio Silva, Cássio Prazeres
Institution: Universidade Federal da Bahia
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7225455246