Work

Label Provenance and the Temporal Degradation of ICU Prediction Models: A Preregistered Equivalence Study

Year: 2026
Type: article
Abstract: Background: Intensive care prediction labels come from rules that read different sources: a creatinine assay, a documented decision to culture and treat, a billing code. We asked whether models trained on labels that depend more on documented clinician action age faster. Methods: Holding cohort, features and task fixed, we ordered four labels by action dependence, transferred three model families from 2008–2013 to three later epochs of MIMIC-IV (51,604 stays) and compared each transferred model with a within-epoch ora-cle. The preregistered estimand, the difference between two labels transfer gaps, was judged against a smallest effect size of interest of 0.010 in the area under the receiver oper-ating characteristic curve. Results: With both tests Holm-adjusted, 4 of 12 estimates are practically equivalent, 6 inconclusive and 2 meaningful in the opposite direction. In 2020–2022, the model for post-hour-6 Sepsis-3 onset aged less than the Sequential Organ Failure Assessment model sharing its trunk (difference −0.0208 [−0.0297, −0.0114], full training set; −0.0227, matched). Its sign persisted in all further checks. Across 168 eICU hospitals, the acute kidney injury contrast is practically equivalent under random hospital splits (−0.0014, full training) but ranges from +0.0481 to −0.0207 by held-out region. Conclusions: The primary analysis found no evidence that models trained on more action-dependent labels age faster.
Language: English
Open Access status: gold
Estimated APC paid: $1,207
OpenAlex ID: W7235106601
FWCI:
Cites:
Cited by:
Related to: 10