Work

Universal Majorization-Minimization Algorithms

Year: 2023
Type: preprint
Abstract: Majorization-minimization (MM) is a family of optimization methods that iteratively reduce a loss by minimizing a locally-tight upper bound, called a majorizer. Traditionally, majorizers were derived by hand, and MM was only applicable to a small number of well-studied problems. We present optimizers that instead derive majorizers automatically, using a recent generalization of Taylor mode automatic differentiation. These universal MM optimizers can be applied to arbitrary problems and converge from any starting point, with no hyperparameter tuning.
Author: Streeter, Matthew
Language: English
Open Access status: green
OpenAlex ID: W4385507657
Cites: 23
Cited by: 7
Related to: 10