Byzantine-Robust Optimization under (L0,L1)-Smoothness
Year: 2026
Type: dissertation
Abstract: We consider distributed optimization under Byzantine attacks in the presence of (L0, L1)-smoothness, a generalization of standard L-smoothness that captures functions with state-dependent gradient Lipschitz constants. We propose Byz-NSGDM, a normalized stochastic gradient descent method with momentum that achieves robustness against Byzantine workers while maintaining convergence guarantees. Our algorithm combines momentum normalization with Byzantine-robust aggregation enhanced by Nearest Neighbor Mixing (NNM) to handle both the challenges posed by (L0, L1)-smoothness and Byzantine adversaries. We prove that Byz-NSGDM achieves a convergence rate of O(K^-1/4), up to a Byzantine bias floor proportional to the robustness coefficient and gradient heterogeneity. Experimental validation on heterogeneous MNIST classification, synthetic (L0, L1)-smooth optimization, and character-level language modeling with a small GPT model demonstrates the effectiveness of our approach against various Byzantine attack strategies. An ablation study further shows that Byz-NSGDM is robust across a wide range of momentum and learning rate choices.
Source: MBZUAI iRep
DOI: 10.82419/345
Author: Bolatov, Arman
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7164173993
Cites:
Cited by:
Related to: