Year: 2026
Type: preprint
Abstract: The subgradient method and its variants are widely used to solve non-smooth minimization problems in machine learning and frequently achieve strong performance in practice. Nevertheless, from a theoretical viewpoint, non-asymptotic convergence analyses of the subgradient method typically rely on weak convexity, a condition that is often violated in realistic machine learning applications. To bridge this gap, we introduce a class of functions that strictly contains the class of weakly convex functions and establish convergence guarantees for the subgradient method within this broader setting. Building on these theoretical results, we reveal two key insights: (i) a polynomially decaying step size is a robust choice for ensuring convergence; (ii) when the loss function exhibits additional structure, such as local smoothness or sharpness, a polynomially decaying step size can be suboptimal compared to constant or geometrically decaying step sizes. These theoretical findings align with empirical observations from prior work and are further validated by our numerical experiments. Overall, our results expand the theoretical applicability of first-order optimization algorithms to a wider class of non-smooth loss functions, bringing mathematical guarantees closer to practical machine learning scenarios.
Source: Research Square
Author: Shunki Arata
Institution: The University of Tokyo
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7236746310
