Work

Scalability of Metropolis-within-Gibbs schemes for high-dimensional Bayesian models

Year: 2025

Type: article

Abstract: We study general coordinate-wise Markov chain Monte Carlo schemes (such as Metropolis-within-Gibbs samplers), which are commonly used to fit Bayesian non-conjugate hierarchical models. We relate their convergence properties to the ones of the corresponding (potentially not implementable) random scan Gibbs sampler through the notion of conditional conductance. This allows us to study the performances of popular Metropolis-within-Gibbs schemes for non-conjugate hierarchical models, in high-dimensional regimes where both number of datapoints and parameters increase. Given random data-generating assumptions, we establish dimension-free convergence results, which are in close accordance with numerical evidences. Application to Bayesian models for binary regression with unknown hyperparameters is also discussed. Motivated by such statistical applications, auxiliary results of independent interest on approximate conductances and perturbation of Markov operators are provided.

Source: Journal of the Royal Statistical Society Series B (Statistical Methodology)

DOI: 10.1093/jrsssb/qkaf084

Authors: Filippo Ascolani, Gareth O Roberts, Giacomo Zanella

Institutions: Duke University, University of Warwick, Bocconi University

Language: English

Topic: Markov Chains and Monte Carlo Methods

Subfield: Statistics and Probability

Field: Mathematics

Domain: Physical Sciences

Open Access status: green

Funders: UK Research and Innovation, Engineering and Physical Sciences Research Council, European Research Council

Awards: G1871434800, G3133235744, DMS-EPSRC: Fast martingales, large deviations and randomised gradients for heavy-tailed target distributions, G6725583053, G7767227200