Year: 2024
Type: preprint
Abstract: Federated Learning (FL) enables edge devices or clients to collaboratively train machine learning (ML) models without sharing their private data. Much of the existing work in FL focuses on efficiently learning a model for a single task. In this paper, we study simultaneous training of multiple FL models using a common set of clients. The few existing simultaneous training methods employ synchronous aggregation of client updates, which can cause significant delays because large models and/or slow clients can bottleneck the aggregation. On the other hand, a naive asynchronous aggregation is adversely affected by stale client updates. We propose FedAST, a buffered asynchronous federated simultaneous training algorithm that overcomes bottlenecks from slow models and adaptively allocates client resources across heterogeneous tasks. We provide theoretical convergence guarantees for FedAST for smooth non-convex objective functions. Extensive experiments over multiple real-world datasets demonstrate that our proposed method outperforms existing simultaneous FL approaches, achieving up to 46.0% reduction in time to train multiple tasks to completion.
Source: arXiv (Cornell University)
Institution: Carnegie Mellon University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: AI Institute for Future Edge Networks and Distributed Intelligence (AI-EDGE), CAREER: Frontiers of Distributed Machine Learning with Communication, Computation and Data Constraints, CPS: Frontier: Software-Defined Nanosatellite Constellations: The Foundation of Future Space-Based Cyber-physical Systems, Collaborative Research: CNS Core: Medium: Data-Centric Networks for Distributed Learning, CAREER: Towards a Marketplace of Networked Services for the Next Billion Devices, G8876996369
OpenAlex ID: W4399355240