Work

Subdifferentially Polynomially Bounded Functions and Gaussian Smoothing–Based Zeroth-Order Optimization

Year: 2025

Type: article

Abstract: . We study the class of subdifferentially polynomially bounded (SPB) functions, which is a rich class of locally Lipschitz functions that encompasses all Lipschitz functions, all gradient- or Hessian-Lipschitz functions, and even some nonsmooth locally Lipschitz functions. We show that SPB functions are compatible with Gaussian smoothing (GS), in the sense that the GS of any SPB function is well-defined and satisfies a descent lemma akin to gradient-Lipschitz functions, with the Lipschitz constant replaced by a polynomial function. Leveraging this descent lemma, we propose GS-based zeroth-order optimization algorithms with an adaptive stepsize strategy for minimizing SPB functions, and we analyze their convergence rates with respect to both relative and absolute stationarity measures. Finally, we also establish the iteration complexity for achieving a [Formula: see text]-approximate stationary point, based on a novel quantification of Goldstein stationarity via the GS gradient that could be of independent interest.

Source: SIAM Journal on Optimization

DOI: 10.1137/24m1659911

Authors: Ming Lei, Ting Kei Pong, Shuqin Sun, Man-Chung Yue

Institutions: Chengdu University of Information Technology, Hong Kong Polytechnic University, China West Normal University, University of Hong Kong

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: Research Grants Council, University Grants Committee

Awards: Specialized Iterative Algorithms for (Distributionally) Robust Optimization, G92056447