Year: 2026
Type: preprint
Abstract: Conformal prediction provides distribution-free uncertainty quantification, but in sequential deployment, standard coverage guarantees can still allow substantial fluctuations in the number of miscoverages over time. We study an online setting in which prediction sets are produced one round at a time and the cumulative number of miscoverages must stay within a prescribed budget throughout deployment.Such a hard coverage guarantee can be enforced by enlarging prediction sets, but how much additional prediction-set size is fundamentally necessary remains unclear. We characterize this unavoidable efficiency cost over $T$ prediction rounds. Remarkably, the cost persists even with i.i.d.\ data, a known score distribution, and perfect knowledge of the population-optimal prediction set. Under suitable regularity conditions relating coverage and prediction-set size, we determine the minimum cumulative excess prediction-set size required by any online method satisfying the hard coverage constraint. This minimum cost is $Θ(\log T)$ for expected cost at a fixed time known in advance, $Θ(T^{1/3})$ for the largest expected cost over all fixed times up to $T$, and $Θ(\sqrt{T})$ for the expected worst realized cost when the evaluation time is chosen in hindsight. We further show that these rates can be attained using past scores, without knowing the score distribution. Thus, even perfect statistical knowledge does not make sequential reliability free.
Source: arXiv (Cornell University)
Author: Liu, Mingrui
Institution: George Mason University
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Funder: National Science Foundation
Awards: Collaborative Research: CAIG: Physics Informed Graph Neural Network-based Snow Water Equivalent Forecasting, Collaborative Research: FDT-BioTech: Advancing Mathematical and Statistical Foundations to Enhance Human Digital Twin of Neurophysiological Modeling and Uncertainty Quantification, Collaborative Research: MFAI: Conformality-Guided Learning with Evolving and Heterogeneous Data
OpenAlex ID: W7236061543
