SelfDistillCore: a federated learning framework with sparse updates and historical knowledge integration
Year: 2026
Type: article
Abstract: Federated learning (FL) offers a robust approach for privacy-preserving model training across distributed clients, but its performance often declines significantly under non-independent and identically distributed data, a critical challenge in real-world scenarios. Existing methods frequently face issues such as slow convergence, reliance on heuristic regularization, or substantial communication overhead. To address these challenges, we propose SelfDistillCore, a federated learning framework that combines a sparsification technique with a momentum-based aggregation rule using an exponentially weighted moving average. By transmitting only the top K% of parameter updates and using momentum to balance historical and new updates, SelfDistillCore mitigates client drift and reduces communication costs. Experimental results show that SelfDistillCore significantly enhances convergence stability, improves resilience to data heterogeneity, increases scalability, and reduces communication costs by 19% to 70% compared to standard baselines. The code of SelfDistillCore can be found at https://github.com/dongld-2020/selfdistillcore_pj .
Source: PeerJ Computer Science
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,395
OpenAlex ID: W7128782844