Exact worst-case coverage of the jackknife+ and a 3/2 law for two-fold cross-validation conformal prediction
Year: 2026
Type: preprint
Abstract: The jackknife+, CV+, and cross-conformal predictors build distribution-free prediction intervals without sacrificing data to a calibration split, but their assumption-free guarantees are essentially coverage 1 − 2α instead of 1 − α. Barber et al. (2021) asked whether the guarantee for K-fold CV+ can be sharpened. We determine the worst case in the two extreme regimes. For the jackknife+, the worst-case miscoverage over all distributions and all symmetric algorithms equals (2⌊α(n + 1)⌋ − 1)₊/(n + 1) for every n and every α ≤ 1/2. The matching construction uses i.i.d. data and an explicit algorithm built from a "zigzag" permutation that realizes a regular tournament through leave-two-out fits. For two-fold CV+ and cross-conformal prediction the factor 2 is not sharp: the worst-case miscoverage converges to (3/2)α as the fold size grows, with explicit non-asymptotic bounds. The reason is that the two fold p-values are asymptotically conditionally i.i.d. given the test point, and the average of two conditionally i.i.d. p-values needs only the factor 3/2. The same sharp inequality gives an exactly valid rule for averaging two independent random splits of split conformal prediction. For general K we give finite-sample bounds that improve both bounds of Barber et al. (2021). We also show that for every fixed K the large-fold worst case is at least α g_K(α), where g_K is the sharp averaging constant of K conditionally i.i.d. p-values. Since g_K → 2, the factor 2 returns as the number of folds grows. The exact worst case for fixed K ≥ 3 remains open. All results are proved, and exact computation is used only to check them.
Author: Yi, Hyunjun
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7221633358
Cites:
Cited by:
Related to: