Work

Quantitative Fluctuation Analysis for Continuous-Time Stochastic Gradient Descent via Malliavin Calculus

Year: 2026

Type: preprint

Abstract: In this paper, we establish a Quantitative Central Limit Theorem (QCLT) for the Stochastic Gradient Descent in Continuous Time (SGDCT) algorithm, whose parameter updates are governed by a stochastic differential equation. We derive an explicit rate at which the SGDCT iterates converge, in the Wasserstein metric, to a critical point of the objective function. This rate is driven primarily by the magnitude of the learning rate: for a fixed convexity constant of the objective function, smaller learning rates lead to slower convergence. Our approach relies on tools from Malliavin calculus. In particular, we apply a second-order Poincaré inequality and obtain explicit bounds by estimating the first- and second-order Malliavin derivatives separately. Controlling the second-order derivative requires several delicate calculations and a careful sequence of decompositions in order to achieve sharp estimates. We complement the theoretical results with several numerical experiments that illustrate the predicted convergence behavior.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2603.07149

Authors: Bourguin, Solesne, Dhama, Shivam Singh, Spiliopoulos, Konstantinos

Institution: Boston University

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Awards: DMS-EPSRC: Asymptotic Analysis of Online Training Algorithms in Machine Learning: Recurrent, Graphical, and Deep Neural Networks, Multiscale Effects and Tail Events for Infinite-Dimensional Processes and Interacting Particle Systems

OpenAlex ID: W7134849596

Cites:

Cited by:

Related to: