Work

A Continuous Time Framework for Discrete Denoising Models

Year: 2022

Type: preprint

Abstract: We provide the first complete continuous time framework for denoising diffusion models of discrete data. This is achieved by formulating the forward noising process and corresponding reverse time generative process as Continuous Time Markov Chains (CTMCs). The model can be efficiently trained using a continuous time version of the ELBO. We simulate the high dimensional CTMC using techniques developed in chemical physics and exploit our continuous time framework to derive high performance samplers that we show can outperform discrete time methods for discrete data. The continuous time treatment also enables us to derive a novel theoretical result bounding the error between the generated sample distribution and the true data distribution.

Source: HAL

DOI: 10.48550/arxiv.2205.14987

Authors: Campbell, Andrew, Benton, Joe, De Bortoli, Valentin, Rainforth, Tom, Deligiannidis, George, Doucet, Arnaud

Institutions: University of Oxford, Centre National de la Recherche Scientifique, École Normale Supérieure - PSL

Language: English

Topic: Generative Adversarial Networks and Image Synthesis

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: U.S. Department of Defense, Defence Science and Technology Laboratory, Multidisciplinary University Research Initiative, Engineering and Physical Sciences Research Council, Defence Science and Technology Group, Office of Defense Programs

Awards: JADE: Joint Academic Data science Endeavour - 2, G2363103010, Semantic Information Pursuit for Multimodal Data Analysis, G5777727104, G6487001647, G8722502606, EPSRC Centre for Doctoral Training in Modern Statistics and Statistical Machine Learning, G926698369, CoSInES (COmputational Statistical INference for Engineering and Security)

OpenAlex ID: W4281626548

Cites: 3

Cited by: 65

Related to: 10