ProtCOVER: Family-Aware Conformal Prediction for Variant Effect Predictors on Unseen Protein Families
Year: 2026
Type: preprint
Abstract: Protein language models and related variant effect predictors (VEPs) increasingly classify missense variants in proteins evolutionarily distant from those used for calibration. Split conformal prediction over variants is valid for its own target, a random variant dominated by well-studied families, but deployment needs coverage for a variant of a new family. On Prote-inGym clinical substitutions with five zero-shot predictors, Mondrian conformal comes within 0.005 of that target for four predictors, yet family-weighted pathogenic coverage is only 0.823– 0.873 at a 0.90 target for all five, without any shift. We present ProtCOVER, a conformal procedure that treats homology families, rather than individual variants, as the exchangeable unit. ProtCOVER (i) calibrates at the family level over label-free homology clusters, giving marginal coverage for a new family under family exchangeability; (ii) adapts to descriptor shift by calibrating conditionally on an estimated density ratio of label-free descriptors (sequence similarity, alignment depth and embedding distance; a variant reweights by the ratio instead), with finite-sample family-weighted coverage, valid as computed, whenever the true ratio is a nonnegative affine function of the estimate; and (iii) can abstain at low effective calibration support, which proved no better than random abstention. Under a shift of its descriptors, Prot-COVER’s mean worse-class shortfall below the target is 0.000, against 0.010 for group-weighted and 0.029 for hierarchical conformal prediction, with larger sets (about equal at matched coverage); on orphan families and held-out embedding regions it matches group weighting (0.002 against 0.003). In evaluations following pre-registered protocols, every shift-aware method met the target on new ClinVar variants of benchmark proteins, which ESM-1b separated better; on 12,225 proteins outside the benchmark, every method fell short for pathogenic variants with either of two predictors, ProtCOVER and a variant conditioned on the descriptors the least, within the 0.01 margin of group weighting.
Institution: Lakehead University
Language: English
Topic: Genomics and Rare Diseases
Subfield: Genetics
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7234399997
