Work

Asynchronous Stochastic Gradient Descent with Model Quantization

Year: 2025

Type: article

Abstract: Distributed optimization is pivotal for large-scale machine learning, yet communication costs remain a significant bottleneck. This paper introduces an asynchronous stochastic gradient descent algorithm augmented with a randomized quantization operator to compress model parameters, thereby substantially reducing communication overhead. Our proposed method maintains convergence guarantees under standard regularity assumptions and achieves near-optimal performance in distributed environments. Comprehensive empirical evaluations demonstrate the algorithm’s efficiency and scalability, validating its effectiveness in practical large-scale learning scenarios.

Source: IFAC-PapersOnLine

DOI: 10.1016/j.ifacol.2025.07.035

Authors: Yubo Du, Keyou You

Institutions: Beijing National Research Center for Information Science and Technology, Tsinghua University

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funder: National Natural Science Foundation of China

Awards: G1850434087, G3518171487, G8836869627