Year: 2014
Type: conference-paper
Abstract: Probabilistic inference on a big data scale is be-coming increasingly relevant to both the machine learning and statistics communities. Here we in-troduce the first fully distributed MCMC algo-rithm based on stochastic gradients. We argue that stochastic gradient MCMC algorithms are particularly suited for distributed inference be-cause individual chains can draw mini-batches from their local pool of data for a flexible amount of time before jumping to or syncing with other chains. This greatly reduces communication overhead and allows adaptive load balancing. Our experiments for LDA on Wikipedia and Pubmed show that relative to the state of the art in distributed MCMC we reduce compute time from 27 hours to half an hour in order to reach the same perplexity level. 1.
Authors: Sungjin Ahn, Babak Shahbaba, Max Welling
Institutions: University of California, Irvine, University of Amsterdam
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
Funder: National Science Foundation
OpenAlex ID: W3401598
