Work

Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support

Year: 2023

Type: preprint

Abstract: The posterior in probabilistic programs with stochastic support decomposes as a weighted sum of the local posterior distributions associated with each possible program path. We show that making predictions with this full posterior implicitly performs a Bayesian model averaging (BMA) over paths. This is potentially problematic, as BMA weights can be unstable due to model misspecification or inference approximations, leading to sub-optimal predictions in turn. To remedy this issue, we propose alternative mechanisms for path weighting: one based on stacking and one based on ideas from PAC-Bayes. We show how both can be implemented as a cheap post-processing step on top of existing inference engines. In our experiments, we find them to be more robust and lead to better predictions compared to the default BMA weights.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2310.14888

Authors: Reichelt, Tim, Ong, Luke, Rainforth, Tom

Institutions: Nanyang Technological University, University of Oxford

Language: English

Topic: Bayesian Modeling and Causal Inference

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: University of Oxford, National Research Foundation, National Research Foundation Singapore, Engineering and Physical Sciences Research Council

Awards: Data-Driven Algorithms for Data Acquisition, EPSRC Centre for Doctoral Training in Autonomous Intelligent Machines and Systems

OpenAlex ID: W4387929614

Cites:

Cited by:

Related to: 10