Application of Optimal Inexact Second-Order Acceleration to Distributed Stochastic Optimization under Statistical Similarity
Year: 2026
Type: preprint
Abstract: We consider distributed stochastic convex optimization with a fixed budget of $N$ independent samples split among $m$ workers. Sample average approximation reduces the problem to a regularized finite-sum problem whose local Hessians are statistically similar. This allows the Hessian of the local objective at the server to be used as an inexact Hessian of the global objective, while the workers communicate only gradients. We apply the optimal accelerated inexact Newton extragradient method of (Chen et al., 2026) and propose its distributed restarted variant for the strongly convex empirical problem. The method reaches the statistical accuracy of order $N^{-1/2}$ in $\widetilde O\left(\max\{N^{1/7},m^{1/4}\}\right)$ communication rounds. Hence, with $m=N^{4/7}$ workers, it requires $\widetilde O\left(N^{1/7}\right)$ rounds, improving the dependence on the total sample size from $\widetilde O\left(N^{1/6}\right)$ for the previous accelerated cubic Newton construction of (Agafonov et al., 2021). Each iteration uses two gradient aggregation rounds and does not require Hessian communication.
Source: arXiv (Cornell University)
Institution: Moscow Institute of Physics and Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Russian Science Foundation
OpenAlex ID: W7213915963