Year: 2024
Type: preprint
Abstract: Diffusion models, which convert noise into new data instances by learning to reverse a diffusion process, have become a cornerstone in contemporary generative modeling. In this work, we develop non-asymptotic convergence theory for a popular diffusion-based sampler (i.e., the probability flow ODE sampler) in discrete time, assuming access to $\ell_2$-accurate estimates of the (Stein) score functions. For distributions in $\mathbb{R}^d$, we prove that $d/\varepsilon$ iterations -- modulo some logarithmic and lower-order terms -- are sufficient to approximate the target distribution to within $\varepsilon$ total-variation distance. This is the first result establishing nearly linear dimension-dependency (in $d$) for the probability flow ODE sampler. Imposing only minimal assumptions on the target data distribution (e.g., no smoothness assumption is imposed), our results also characterize how $\ell_2$ score estimation errors affect the quality of the data generation processes. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach without the need of resorting to SDE and ODE toolboxes.
Source: arXiv (Cornell University)
Authors: Li, Gen, Wei, Yuting, Chi, Yuejie, Chen, Yuxin
Institutions: Chinese University of Hong Kong, University of Pennsylvania
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
Awards: G1056225131, G1469985387, G1523888516, CAREER: Statistical Learning from a Modern Perspective: Over-parameterization, Regularization, and Generalization, Collaborative Research: CIF: Medium: Statistical and Algorithmic Foundations of Efficient Reinforcement Learning, Taming Nonlinear Inverse Problems: Theory and Algorithms, CIF: Small: Non-Asymptotic Foundations of Approximate Message Passing, G4720003262, Collaborative Research: CIF: Medium: Statistical and Algorithmic Foundations of Efficient Reinforcement Learning, CIF: Small: Taming Nonconvexity in High-Dimensional Statistical Estimation, G6098521345, G6671297155, G8876996369, Collaborative Research: Towards a Theoretic Foundation for Optimal Deep Graph Learning
OpenAlex ID: W4403369890