Variational Inference with Rényi Divergence.
Year: 2016
Type: preprint
Abstract: We introduce the variational R\'enyi bound (VR) that extends traditional variational inference to R\'enyi's alpha-divergences. This new family of variational lower-bounds unifies a number of existing variational methods, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of alpha. The reparameterization trick, Monte Carlo estimation and stochastic optimisation methods are deployed to obtain a unified implementation for the VR bound optimisation. We further consider negative alpha values and propose a novel variational inference method as a new special case in the proposed framework. Experiments on variational auto-encoders and Bayesian neural networks demonstrate the wide applicability of the VR bound.
Source: arXiv (Cornell University)
Authors: Yingzhen Li, Richard E. Turner
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2271421239