Work

On the SAGA Algorithm with Decreasing Step

Year: 2026
Type: article
Abstract: Stochastic optimization naturally appears in many application areas, including machine learning. Our goal is to go further in the analysis of the stochastic average gradient accelerated (SAGA) algorithm. To achieve this, we introduce a new [Formula: see text]-SAGA algorithm which interpolates between the stochastic gradient descent ([Formula: see text]) and the SAGA algorithm ([Formula: see text]). First, we investigate the almost sure convergence of this new algorithm with decreasing step which allows us to avoid the restrictive strong convexity and Lipschitz gradient hypotheses associated with the objective function. Second, we establish a central limit theorem for the [Formula: see text]-SAGA algorithm. Finally, we provide the nonasymptotic [Formula: see text] rates of convergence.
Language: English
Open Access status: green
OpenAlex ID: W4403116298
FWCI:
Cites: 38
Cited by: 1
Related to: 10