Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference
Year: 2025
Type: article
Abstract: The application of artificial intelligence (AI) models in fields such as engineering is limited by the known difficulty of quantifying the reliability of an AI’s decision. A well-calibrated AI model must correctly report its accuracy on indistribution (ID) inputs, while also enabling the detection of out-of-distribution (OOD) inputs. Conventional solutions are effective at addressing only one of these two conflicting requirements. This paper proposes a novel solution that endows Bayesian learning with enhanced ID calibration and OOD detection capabilities by integrating calibration regularization for improved ID performance, confidence minimization for OOD detection, and selective calibration to ensure a synergistic use of calibration regularization and confidence minimization. Selective calibration rejects inputs for which the calibration performance is expected to be insufficient, supporting effective OOD detection, while also ensuring ID calibration. Prior art had only considered these ideas in isolation and for frequentist learning. Numerical results illustrate the trade-offs between ID accuracy, ID calibration, and OOD calibration, showing that the proposed novel Bayesian approach achieves the best ID and OOD performance compared to existing state-of-the-art approaches, at the cost of rejecting a fraction of the inputs.
Authors: Jiayi Huang, Sangwoo Park, Osvaldo Simeone
Institution: King's College London
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4416078055