Work

Block Majorization-Minimization with Diminishing Radius for Constrained Nonsmooth Nonconvex Optimization

Year: 2025
Type: article
Abstract: Block majorization-minimization (BMM) is a simple iterative algorithm for constrained nonconvex optimization that sequentially minimizes majorizing surrogates of the objective function in each block while the others are held fixed. BMM entails a large class of optimization algorithms such as block coordinate descent and its proximal-point variant, expectation-minimization, and block projected gradient descent. We first establish that for general constrained nonsmooth nonconvex optimization, BMM with [Formula: see text]-strongly convex and [Formula: see text]-smooth surrogates can produce an [Formula: see text]-approximate first-order optimal point within [Formula: see text] iterations and asymptotically converges to the set of first-order optimal points. Next, we show that BMM combined with a trust-region method with diminishing radius has an improved complexity of [Formula: see text], independent of the inverse strong convexity parameter [Formula: see text], allowing improved theoretical and practical performance with “flat” surrogates. Our results hold robustly even when the convex subproblems are solved inexactly as long as the optimality gaps are summable. Central to our analysis is a novel continuous first-order optimality measure, by which we bound the worst-case suboptimality in each iteration by the first-order improvement the algorithm makes. We apply our general framework to obtain new results on various algorithms such as the celebrated multiplicative update algorithm for nonnegative matrix factorization by Lee and Seung, regularized nonnegative tensor decomposition, and the classical block projected gradient descent algorithm. Lastly, we numerically demonstrate that the additional use of diminishing radius can improve the convergence rate of BMM in many instances.
Language: English
Open Access status: green
OpenAlex ID: W4409805047
FWCI: 2.345
Cites: 65
Cited by: 33
Related to: 8