Fast Gradient Algorithm with Dry-like Friction and Nonmonotone Line Search for Nonconvex Optimization Problems
Year: 2024
Type: article
Abstract: . In this paper, we propose a fast gradient algorithm for the problem of minimizing a differentiable (possibly nonconvex) function in Hilbert spaces. We first extend the dry friction property for convex functions to what we call the dry-like friction property in a nonconvex setting, and then employ a line search technique to adaptively update parameters at each iteration. Depending on the choice of parameters, the proposed algorithm exhibits subsequential convergence to a critical point or full sequential convergence to an “approximate” critical point of the objective function. We also establish the full sequential convergence to a critical point under the Kurdyka–Łojasiewicz (KL) property of a merit function. Thanks to the parameters’ flexibility, our algorithm can reduce to a number of existing inertial gradient algorithms with Hessian damping and dry friction. By exploiting variational properties of the Moreau envelope, the proposed algorithm is adapted to address weakly convex nonsmooth optimization problems. In particular, we extend the result on KL exponent for the Moreau envelope of a convex KL function to a broad class of KL functions that are not necessarily convex nor continuous. Simulation results illustrate the efficiency of our algorithm and demonstrate the potential advantages of combining dry-like friction with extrapolation and line search techniques.
Source: SIAM Journal on Optimization
DOI: 10.1137/22m1532354
Authors: Lien T. Nguyen, Andrew Eberhard, Xinghuo Yu, Chaojie Li
Institutions: RMIT University, UNSW Sydney
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Australian Research Council
OpenAlex ID: W4400740271