An Adaptive Augmented Lagrangian Method for Deterministic and Stochastic Nonconvex Optimization
Year: 2026
Type: preprint
Abstract: We present an inexact Augmented Lagrangian algorithm for solving nonlinear, non-convex optimization problems. Unlike most recently proposed Augmented Lagrangian methods with worst-case complexity guarantees, we utilize adaptive penalty parameter updates and full dual stepsizes. We show that the method matches the best known worst-case complexity results for Augmented Lagrangian methods (up to logarithmic factors) when both the function and constraints are deterministic, when the function is stochastic and the constraints are deterministic, and when both are stochastic. Experiments on CUTEst test problems confirm the practical advantages of the proposed approach over Augmented Lagrangian methods with non-adaptive penalty parameters and/or short dual step sizes in the deterministic setting. Numerical results on stochastic constrained optimization problems in machine learning also confirm these findings.
Source: arXiv (Cornell University)
Authors: Liu, Tianzhu, O'Neill, Michael J.
Subfield: Numerical Analysis
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204766694
Cites:
Cited by:
Related to: