Efficient Convex Optimization Requires Superlinear Memory
Year: 2024
Type: article
Abstract: We show that any memory-constrained, first-order algorithm which minimizesd-dimensional, 1-Lipschitz convex functions over the unit ball to 1/poly(d) accuracy using at most d1.25 - δbits of memory must make at least \(\tilde{\Omega }(d^{1 + (4/3)\delta })\) first-order queries (for any constant \(\delta \in [0, 1/4]\) ). Consequently, the performance of such memory-constrained algorithms are at least a polynomial factor worse than the optimal Õ(d) query bound for this problem obtained by cutting plane methods that use Õ(d2) memory. This resolves one of the open problems in the COLT 2019 open problem publication of Woodworth and Srebro.
Source: Journal of the ACM
DOI: 10.1145/3689208
Institutions: Stanford University, University of Southern California
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: The Stanford Data Science Collaboratory, AF:Medium:Collaborative Research:Estimation, Learning, and Memory: The Quest for Statistically Optimal Algorithms, G2267761032, CAREER: Theory of Fast Graph Optimization, G5972442391, G6671297155, CAREER: Computational Foundations of Modern Machine Learning, AF: Small: Robust and Secure Learning, Collaborative Research: AF: Medium: Foundations of Structured Optimization, AF: Small: Memory Bounded Optimization and Learning, G8876996369
OpenAlex ID: W4402310499