Work

Heavy-ball differential equation achieves $$O(\varepsilon ^{-7/4})$$ convergence for nonconvex functions

Year: 2026

Type: article

Abstract: First-order optimization methods for nonconvex functions with Lipschitz continuous gradient and Hessian have been extensively studied. State-of-the-art methods for finding an $$\varepsilon $$ -stationary point within $$O(\varepsilon ^{-{7/4}})$$ or $$\tilde{O}(\varepsilon ^{-{7/4}})$$ gradient evaluations are based on Nesterov’s accelerated gradient descent (AGD) or Polyak’s heavy-ball (HB) method. However, these algorithms employ additional mechanisms, such as restart schemes and negative curvature exploitation, which complicate their behavior and make it challenging to apply them to more advanced settings (e.g., stochastic optimization). As a first step in investigating whether a simple algorithm with $$O(\varepsilon ^{-{7/4}})$$ complexity can be constructed without such additional mechanisms, we study the HB differential equation, a continuous-time analogue of the AGD and HB methods. We prove that its dynamics attain an $$\varepsilon $$ -stationary point within $$O(\varepsilon ^{-{7/4}})$$ time.

Source: Optimization Letters

DOI: 10.1007/s11590-025-02270-w

Authors: Kaito Okamura, Naoki Marumo, Akiko Takeda

Institutions: The University of Tokyo, RIKEN, RIKEN Center for Advanced Intelligence Project

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $3,090

Funders: University of Tokyo, Japan Society for the Promotion of Science, Core Research for Evolutional Science and Technology