Work

Accelerated Objective Gap and Gradient Norm Convergence for Gradient Descent via Long Steps

Year: 2025

Type: article

Abstract: This work considers gradient descent for L-smooth convex optimization with stepsizes larger than the classic regime where descent can be ensured. The stepsize schedules considered are similar to but differ slightly from the recent silver stepsizes of Altschuler and Parrilo. For one of our stepsize sequences, we prove a [Formula: see text] convergence rate in terms of objective gap decrease and for the other, we show the same rate of decrease for squared-gradient-norm decrease. This first result improves on the recent result of Altschuler and Parrilo by a constant factor, while the second results improve on the exponent of the prior best squared-gradient-norm convergence guarantee of [Formula: see text]. Funding: B. Grimmer’s work was supported in part by the Air Force Office of Scientific Research [Award FA9550-23-1-0531].

Source: INFORMS Journal on Optimization

DOI: 10.1287/ijoo.2024.0057

Authors: Benjamin Grimmer, Kevin Shu, Alex L. Wang

Institutions: Johns Hopkins University, California Institute of Technology, Purdue University West Lafayette

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

OpenAlex ID: W4408326507

FWCI: 1.632

Cites: 8

Cited by: 4

Related to: 10