Seeded Probes Do Not Identify a Reviewer’s Miss Rate: One-Sided Misclassification in Distribution-Free Risk Control
Year: 2026
Type: preprint
Abstract: Distribution-free risk control certifies that the rate of defects that would change clinical management, among the answers an AI system releases, stays below a stated level, calibrated on cases a human reviewer has checked. The reviewer is an imperfect reference standard whose errors run one way: a missed defect enters calibration as a safe label. This one-sided misclassification lowers the risk on record at every release threshold, so the certificate is anti-conservative by a factor set by the reviewer’s miss rate on the system’s own defects. We show that seeded probes, which plant known defects and count the catches, do not identify this miss rate. A reviewer’s chance of missing a defect depends on how hard that defect is to spot, its detectability. A probe samples defects at detectabilities its designer chose. The system’s own defects fall at detectabilities nothing chose, and converting the probe’s miss rate into theirs requires the very rate being sought. This holds whenever detectability is not fully captured by an observed score, since a score that captured it would let probes covering its range identify the rate. Cytology field validation builds probe sets whose miss rate is a floor on the true one. Even then the probe bounds the rate from below only, so it can reject a certificate and cannot grant one. The finite-sample label-noise corrections assume noise independent of the case given its label, but a reviewer misses the defects that read as plausible, and the condition they need fails in all four simulated reviewer profiles we examine. Testing that condition on an independent sample takes 185 adjudicated cases for a reviewer missing 85% of defects and 147,938 for one missing 3%. A second reader who adjudicates the cases the careful reviewer passed finds a miss with 95% confidence within 498, so looking for the noise directly costs far less than testing a model of it. In a public corpus of model answers graded by physicians, a physician passed 26.3% of the defects the other graders agreed were there (95% interval 21.6% to 31.7%, resampling whole conversations), on criteria about seeking context and stating uncertainty, and individual rates ran from 8.0% to 54.5%.
Source: medRxiv
Language: English
Subfield: Safety, Risk, Reliability and Quality
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7216292385