On the Complexity of Finding Stationary Points in Stochastic $(L_0, L_1)$-Smooth Nonconvex Optimization
Year: 2026
Type: preprint
Abstract: We study the stochastic first-order oracle (SFO) complexity of finding $ε$-stationary points of $(L_0,L_1)$-smooth nonconvex functions. Under bounded-variance noise, existing analyses of normalized and clipped stochastic gradient descent (SGD) establish optimal dependence on $ε$ and $L_0$, but the jointly optimal dependence on $L_0$, $L_1$, and $ε$ remains unclear. We resolve this question for an unbiased SFO whose gradient noise has a $p$th central moment bounded by $σ^p$ for some $σ\geq 0$, where $p\in(1,2]$.Specifically, we establish an SFO lower bound of $Ω\big(\big(σ^{\frac{p}{p-1}}ε^{-\frac{p}{p-1}}+1\big) (L_0 Δε^{-2}+L_1Δε^{-1})\big)$, where $Δ>0$ is the initial optimality gap.This bound identifies an unavoidable contribution proportional to $L_1ε^{-1}Δ$. We also prove that mini-batch (smoothed) normalized SGD can attain a matching upper bound, thereby establishing the optimal SFO complexity.
Source: arXiv (Cornell University)
Authors: Liang, Yanpeng, Luo, Luo
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7222750316