Work

Federated ADMM from Bayesian Duality

Year: 2025

Type: preprint

Abstract: We propose a new Bayesian approach to generalize the federated Alternating Direction Method of Multipliers (ADMM). We show that the solutions of variational-Bayesian (VB) objectives are associated with a duality structure that not only resembles the structure of ADMM's fixed-points but also generalizes it. For example, ADMM-like updates are recovered when the VB objective is optimized over the isotropic-Gaussian family, and new non-trivial extensions are obtained for other exponential-family distributions. These extensions include a Newton-like variant that converges in one step on quadratic objectives and an Adam-like variant that yields up to 7% accuracy boosts for deep heterogeneous cases. Our work opens a new Bayesian way to generalize ADMM and other primal-dual methods.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2506.13150

Authors: Möllenhoff, Thomas, Swaroop, Siddharth, Doshi-Velez, Finale, Khan, Mohammad Emtiyaz

Institutions: RIKEN, RIKEN Center for Advanced Intelligence Project, University College London, Harvard University

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, Core Research for Evolutional Science and Technology

Awards: HCC: Medium: Improving Human-AI Collaboration on Decision-Making Tasks, G5613623313

OpenAlex ID: W4415108083

Cites:

Cited by:

Related to: