Auditing Machine Learning for Acute Myocardial Infarction: Leakage, Calibration, Label Circularity, and Explainable ECG Evidence
Year: 2026
Type: preprint
Abstract: Machine-learning risk scores for cardiovascular disease are proliferating, yet independent appraisals repeatedly find the same three defects: silent data leakage, an absent calibration analysis, and no clinician involvement in defining the outcome being predicted. A model can therefore report a near-perfect area under the curve and still be unusable at the bedside. This paper presents TRUST-AMI, a leakage-audited, calibration-aware, and explainable approach for one narrowly scoped and time-critical decision: the rapid rule-in and rule-out of acute myocardial infarction (AMI) from routinely collected cardiac biomarkers, supported by a separately audited electrocardiogram (ECG) imaging branch. A cohort of 1,319 emergency presentations (positive = 810, negative = 509) described by eight bedside variables was analysed under a protocol in which imputation and scaling are fitted inside cross-validation folds, model selection uses 5-fold stratified cross-validation on the training partition alone, and the held-out test set (n = 264) is scored exactly once. Four classifiers were compared across calibration, clinical operating characteristics, 1,000-sample bootstrap intervals, ablation, decision-curve analysis, and sex and age subgroup analysis. The selected random forest attained a test ROC-AUC of 0.996 (95% CI 0.988–1.000), sensitivity 1.000, specificity 0.961, and agreement with the cardiologist reference of κ = 0.968. The ECG branch, trained on PTB-XL under patient-disjoint folds, reached a test ROC-AUC of 0.908 (n = 1,517) with Grad-CAM supplying waveform-level evidence. A second, clinically validated four-class ECG image corpus was audited before use, and the audit is itself a result: its 928 released files contain 491 distinct images and 311 distinct patients, and 87.4% of the acute-MI class consists of byte-identical copies. Holding estimator, features, and corpus fixed and varying only what the partition may share, macro-F1 falls from 0.969 to 0.656 and acute-MI F1 from 0.998 to 0.121. The resulting tool is an AMI triage aid within coronary heart disease and is not presented as a general cardiovascular-disease diagnoser. All code, seeds, and results are released publicly, and the study is offered as a reproducible template for reporting clinical prediction models transparently.
Source: Research Square
Language: English
Subfield: Health Information Management
Field: Health Professions
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7225795758