Year: 2022
Type: preprint
Abstract: Diffusion models have proven to be a flexible and effective framework for modelling probability distributions on finite-dimensional spaces. However, many physical modelling problems such as time series are naturally described over function spaces. In this work we apply diffusion models to such stochastic processes. To do so we consider a spectral representation of the data, obtained using a kernel, thereby dissociating the stochastic part of the processes from their space-time structure. As a result, the stochasticity of the processes is entirely encoded in the spectral coefficients, which we truncate and model using standard finite-dimensional diffusion models. By truncating the representation in the spectral domain we ensure our resulting model defines valid stochastic processes, thereby naturally satisfying consistency and exchangeability criteria. Projecting our spectral diffusion models back to the original input space, we show that for any given marginals our approach corresponds to a diffusion model with correlated noise, with explicit covariance matrix given by the kernel. We demonstrate our method's effectiveness for modelling various multimodal datasets as well as conditional sampling by amortising our models with respect to a context set.
Authors: Phillips, Angus, Seror, Thomas, Hutchinson, Michael, De Bortoli, Valentin, Doucet, Arnaud, Mathieu, Emile
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1871434800, Semantic Information Pursuit for Multimodal Data Analysis, G5777727104, G6194403621, Scalable Qubit Arrays for Quantum Computing and Optimisation, G8722502606, EPSRC Centre for Doctoral Training in Modern Statistics and Statistical Machine Learning, G926698369, CoSInES (COmputational Statistical INference for Engineering and Security)
OpenAlex ID: W4298055074