Work

SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives

Year: 2014

Type: preprint

Abstract: In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and has support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem. We give experimental results showing the effectiveness of our method.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.1407.0202

Authors: Defazio, Aaron, Bach, Francis, Lacoste-Julien, Simon

Institutions: Institut national de recherche en sciences et technologies du numérique, Data61, École Normale Supérieure - PSL, SIERRA: Apprentissage Statistique et optimisation

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W2952215077

Cites: 14

Cited by: 1,287

Related to: 9