Failure Modes of Prediction Models Built from Electronic Health Record Data: Methods and Preliminary Results of the First Pass of a Scoping Review
Year: 2026
Type: preprint
Study design: Systematic Review
Abstract: This report records the methods and preliminary results of the first pass of a scoping review of failure modes of prediction models built from electronic health record (EHR) data. A failure mode is a distinct way in which a failure can arise at any stage of a prediction project. A codebook of 80 anticipated failure modes in 14 families was built from three independent sources: a process map derived from a draft theory of how EHR data are created and reused, reporting standards and appraisal tools, and a random sample of the literature. PubMed was searched for records added on or before 23 September 2026, and 340 records were screened on title and abstract by one reviewer with advisory labels from a large language model. In all, 192 papers were provisionally included (86 central, 32 peripheral and 74 on mitigation). They mention 53 of the 80 anticipated failure modes, 47 of them in a central paper, and suggest seven provisional new modes. An earlier mapping of the failure modes onto the draft process map suggested that it could explain most of them at least in part and identified where it falls short. All results are preliminary and will be revised when the full texts are read in dedicated reviews of each family.
Source: OSF Preprints (OSF Preprints)
Author: Uday Pratap Singh
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225917254
