Year: 2023
Type: preprint
Abstract: We study the problem of training a flow-based generative model, parametrized by a two-layer autoencoder, to sample from a high-dimensional Gaussian mixture. We provide a sharp end-to-end analysis of the problem. First, we provide a tight closed-form characterization of the learnt velocity field, when parametrized by a shallow denoising auto-encoder trained on a finite number $n$ of samples from the target distribution. Building on this analysis, we provide a sharp description of the corresponding generative flow, which pushes the base Gaussian density forward to an approximation of the target density. In particular, we provide closed-form formulae for the distance between the mean of the generated mixture and the mean of the target mixture, which we show decays as $Θ_n(\frac{1}{n})$. Finally, this rate is shown to be in fact Bayes-optimal.
Source: arXiv (Cornell University)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1472743852, DMS-EPSRC Collaborative Research: Sharp Large Deviation Estimates of Fluctuations in Stochastic Hydrodynamic Systems, OPtimal Estimation in RAndom Generative mOdelS: Theory, Thresholds, Tractability, NYU Materials Research Science and Engineering Center, Statistical and Computational Foundations of Deep Generative Models, G821361635, Solvable Models of Learning with Artificial Neural Networks
OpenAlex ID: W4387433471