Three-Layer Calibration for Bayesian Robust Variance Estimation in Meta-Analysis With Dependent Effect Sizes
Year: 2026
Type: preprint
Abstract: Meta-analyses in education, psychology, and medicine routinely extract several statistically dependent effect sizes from each study, and the within-study correlations needed to model that dependence are rarely reported. Frequentist practice resolves this through robust variance estimation: working models paired with cluster-robust standard errors, bias-reduced small-sample corrections, and Satterthwaite degrees of freedom. Bayesian meta-analysis offers complementary strengths (principled shrinkage, flexible priors on heterogeneity, and direct probability statements), but its posterior spread inherits whatever working dependence model the analyst assumes, and a Bayesian counterpart that calibrates this spread to the robust sandwich covariance has so far been lacking. This article introduces ThreeLayer Calibration, a calibrated-Bayes procedure for Bayesian meta-regression with dependent effect sizes. Layer 1 retains the posterior mean as the point estimate. Layer 2 applies a Cholesky Calibration Correction that affinely transforms posterior draws so their sample covariance matches a bias-reduced (CR2) cluster sandwich exactly in finite samples while preserving the posterior mean. Layer 3 replaces the normal reference distribution with per-coefficient Satterthwaite t distributions. In a 324-condition simulation built on a real meta-analysis, calibrated intervals attained 0.952 coverage for the average effect and uniform 0.966–0.978 across slopes, which carry roughly one third the intercept’s degrees of freedom. A normal-reference variant undercovered all three slopes (0.908–0.931); the frequentist benchmark undercovered only the smallest (0.905), where the largest gain occurs, at a documented power cost. A reanalysis of 1,198 effect sizes from 117 studies of brief alcohol interventions illustrates the workflow. Methods are implemented in the R package bayesRVE.
Authors: Lee, JoonHo, Tipton, Elizabeth
Institutions: University of Alabama, Northwestern University
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7204274900
Cites:
Cited by:
Related to: