Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning
Year: 2025
Type: preprint
Abstract: Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its scalability is limited by synchronization overhead. Asynchronous federated learning (AFL) alleviates this issue by allowing clients to communicate independently, thereby improving wall-clock efficiency in large-scale, hardware-heterogeneous environments. However, asynchrony introduces updates computed on outdated global models (staleness) that can destabilize optimization and hinder convergence. We propose FedRevive, an AFL framework that revives stale updates through data-free knowledge distillation (DFKD). FedRevive integrates parameter-space aggregation with a server-side DFKD process that transfers knowledge from stale client updates to the current global model without access to data. A meta-learned generator synthesizes pseudo-samples for multi-teacher distillation. A hybrid aggregation scheme combining raw with DFKD updates effectively mitigates staleness while retaining AFL scalability. Experiments on vision and text benchmarks show that FedRevive achieves faster training by up to 29.7% and higher final accuracy by up to 11.4 percentage points than baselines.
Source: arXiv (Cornell University)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: AI Institute for Future Edge Networks and Distributed Intelligence (AI-EDGE), SCC-IRG Track 2: NeighborDrive: Community-driven Neighborhood Wellbeing Improvement through Active Vehicular Crowdsensing, CAREER: Frontiers of Distributed Machine Learning with Communication, Computation and Data Constraints, CPS: Frontier: Software-Defined Nanosatellite Constellations: The Foundation of Future Space-Based Cyber-physical Systems, Collaborative Research: CNS Core: Medium: Data-Centric Networks for Distributed Learning
OpenAlex ID: W4415938577