Year: 2025
Type: preprint
Abstract: The Barzilai-Borwein (BB) step sizes have a profound impact on gradient descent methods. In this work, we propose two new gradient step sizes: one longer than the original long BB step size, and the other shorter than the original short BB step size. This extends the bounds of the original BB step sizes. For strictly convex quadratic optimization problems, we employ the dynamics of difference equations to prove that these two new methods achieve R-linear convergence. Regarding stability, we surprisingly find that under certain conditions, the gradient descent method corresponding to the longer step size is stable, whereas the shorter step size consistently leads to instability. Numerical results validate these stability theories. Here, stability refers to whether the gradient norm decreases monotonically.
Source: arXiv (Cornell University)
Author: Xu, Xin
Language: English
Subfield: Numerical Analysis
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4414456924
Cites:
Cited by:
Related to: