Year: 2019
Type: article
Abstract: This note discusses proofs of convergence for gradient methods (also called “first-order methods”) based on simple potential-function arguments. We cover methods like gradient descent (for both smooth and non-smooth settings), mirror descent, and some accelerated variants. We hope the structure and presentation of these amortized-analysis proofs will be useful as a guiding principle in learning and using these proofs.
Source: Theory of Computing
Authors: Nikhil Bansal, Anupam Gupta
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
Awards: G1664046972, CCF-BSF: AF: Small: Metric Embeddings and Partitioning for Minor-Closed Graph Families, RCN: DIMACS/Simons Collaboration on Bridging Continuous and Discrete Optimization, Continuous Methods in Discrete Optimization, AitF: FULL: Collaborative Research: Practical Foundations for Software-Defined Network Optimization, BSF: 2014414: New Challenges and Perspectives in Online Algorithms, Algorithms for coping with uncertainty and intractability
OpenAlex ID: W2981349516
