Rapid Bayesian computation and estimation for neural networks via log-concave coupling
Year: 2026
Type: article
Abstract: This paper presents the study of a Bayesian estimation procedure for single-hidden-layer neural networks using \ell_{1} controlled neuron weight vectors. We study the structure of the posterior density and provide a representation that makes it amenable to rapid sampling via Markov Chain Monte Carlo (MCMC). Let the neural network have K neurons with internal weights of dimension d and fix the outer weights. Thus, there are Kd parameters overall. With N data observations, use a gain parameter or inverse temperature of \beta in the posterior density for the internal weights. The posterior is intrinsically multi-modal and not naturally suited to rapid mixing of direct MCMC algorithms. For a continuous uniform prior on the \ell_{1} ball, we demonstrate that the posterior density can be written as a mixture density with suitably defined auxiliary random variables, where the mixture components are log-concave. Furthermore, when the total number of model parameters Kd is large enough that Kd \geq C(\beta N)^{2} , the mixing distribution of the auxiliary random variables is also log-concave. Thus, neuron parameters can be sampled from the posterior by only sampling log-concave densities. The authors refer to the pairing of weights with such auxiliary random variables as a log-concave coupling.
DOI: 10.4171/msl/59
Authors: Curtis McDonald, Andrew R. Barron
Institutions: University of California, Berkeley, Yale University
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7202289046