d-VMP: Distributed Variational Message Passing
Year: 2016
Type: conference-paper
Abstract: Motivated by a real-world financial dataset, we propose a distributed variational message passing scheme for learning conjugate exponential models.We show that the method can be seen as a projected natural gradient ascent algorithm, and it therefore has good convergence properties.This is supported experimentally, where we show that the approach is robust wrt.common problems like imbalanced data, heavy-tailed empirical distributions, and a high degree of missing values.The scheme is based on map-reduce operations, and utilizes the memory management of modern big data frameworks like Apache Flink to obtain a time-efficient and scalable implementation.The proposed algorithm compares favourably to stochastic variational inference both in terms of speed and quality of the learned models.For the scalability analysis, we evaluate our approach over a network with more than one billion nodes (and approx.75% latent variables) using a computer cluster with 128 processing units.
Authors: Andrés R. Masegosa, Ana María Martínez, Helge Langseth, Thomas D. Nielsen, Antonio Salmerón, Darío Ramos-López, Anders L. Madsen
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: European Commission
OpenAlex ID: W2520167924
