Statistical frameworks for reliable machine learning predictions and inference
Year: 2026
Type: article
Abstract: Machine Learning (ML) models often achieve high accuracy, but fail to meet the reliability and robustness standards required for Official Statistics (OS). Neural networks, in particular, function as black-box predictors prone to overconfidence, offering no direct method to measure true uncertainty in predictions and estimates of population parameters. Non-rigorous approaches include treating ML predictions as gold standard data and heuristic notions of uncertainty, like softmax scores in classification problems, as valid measures of confidence. This can easily lead to unreliable uncertainty quantification. This paper handles two distinct problems: (1) quantifying prediction-level uncertainty for new observations and (2) quantifying noise-free uncertainty for estimates of population parameters. We propose handling the former via conformal prediction (CP) and the latter using prediction-powered inference (PPI). Both are model-agnostic statistical frameworks for uncertainty quantification. Finally, we present real-world use cases for OS, applying both techniques.
Source: Statistical Journal of the IAOS
Authors: Francesco Ortame, Francesco Isidori
Institutions: Istituto Nazionale di Statistica, Bank of Italy
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,900
OpenAlex ID: W7128814520