The logarithmic factor in the tuned constant-step rate of random reshuffling
Year: 2026
Type: preprint
Abstract: For strongly convex finite sums, the known last-iterate upper bounds for random reshuffling with a constant step size chosen from the number of epochs carry a logarithmic factor, while the known lower bounds carry none. We show that for constant steps the factor cannot be removed. On a one-dimensional sum of $n_1$ quadratic and $n-n_1$ linear components, with average curvature $\mu$, gradient bound $G$ at the minimiser and initial distance $D$, every constant step $\gamma\le 1/L$ leaves $\mathbb{E} F(x_K)-F^*\ge G^2(\Lambda-\ln\Lambda)^2/(512\,\mu T^2)$ after $K$ epochs, where $T=nK$ and $\Lambda=\ln(\mu TD/G)$, while the step $\gamma=\Lambda/(\mu T)$ gives at most $G^2\Lambda^2(1+o(1))/(8\mu T^2)$ once $K$ is large compared with $n_1\Lambda$. The proof computes one epoch exactly: the stationary mean of the epoch-end iterate is $\gamma G\varphi/2$, where $\varphi$ is a weighted mean lying between $(1-\gamma L)^{n_1-1}$ and $1$, and the epoch variance reduces to the variance of a Mann–Whitney statistic. The bound concerns the worst starting point: from the other side of the minimiser a tuned step can cancel the transient against the bias, while a start of random sign keeps the bound for every step.
Author: Shmalo, Yitzchak
Institution: Hebrew University of Jerusalem
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225620998
Cites:
Cited by:
Related to: