Year: 2022
Type: preprint
Abstract: Black-box variational inference performance is sometimes hindered by the use of gradient estimators with high variance. This variance comes from two sources of randomness: Data subsampling and Monte Carlo sampling. While existing control variates only address Monte Carlo noise, and incremental gradient methods typically only address data subsampling, we propose a new "joint" control variate that jointly reduces variance from both sources of noise. This significantly reduces gradient variance, leading to faster optimization in several applications.
Source: arXiv (Cornell University)
Authors: Wang, Xi, Geffner, Tomas, Domke, Justin
Institution: University of Massachusetts Amherst
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
Funder: National Science Foundation
OpenAlex ID: W4320536651