Year: 2026
Type: article
Abstract: In the paradigm of decentralized learning, a group of agents collaborate to learn a global model using a distributed dataset without a central server; nevertheless, it is severely challenged by the heterogeneity of the data distribution across the agents. For example, the data may be distributed non-independently and identically, and even be noised or poisoned. To address these data challenges, we propose ROSS, a novel robust decentralized stochastic learning algorithm based on Shapley values, in this paper. Specifically, in each round, each agent leverages its local gradient and the cross-gradients from its neighbors (i.e., the derivative of its local loss function and the derivatives of its neighbors’ local loss functions evaluated at its local model) to update its local model in a momentum-like manner, while we innovate in weighting the derivatives according to their contributions measured by Shapley values. We perform solid theoretical analysis to reveal the linear convergence speedup of our ROSS algorithm. We also verify the efficacy of our algorithm through extensive experiments on public datasets. Our results demonstrate that, in face of the above variety of data challenges, our ROSS algorithm have oblivious advantages over existing state-of-the-art proposals in terms of both convergence and prediction accuracy.
Source: IEEE Transactions on Networking
Authors: Lina Wang, Yunsheng Yuan, Feng Li, Lingjie Duan
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G3565984400, G514600894, G8079565522
OpenAlex ID: W7124458905