Work

Machine Learning vs. Clinical Judgement for Extubation Failure: A Rigorous Re-evaluation

Year: 2026
Type: preprint
Abstract: Background In 2022 we derived and validated the ABCDMed score to predict extubation failure. Machine learning models (MLM) for the same outcome now report areas under the receiver operating characteristic curve (AUC) of up to 0.98. We asked whether flexible algorithms, given the same patients may improve on a model built from clinical judgement. Methods. A secondary analysis of a original prospective cohort of 800 consecutive adults extubated on a ICU in Bogotá, Colombia, at 2,600 m above sea level was performed. Extubation failure prediction was compared with two MLM, random forest, gradient boosting, and with nine further strategies, under 10 times repeated stratified 5-fold cross validation in which imputation, scaling, resampling, variable selection and hyperparameter search were performed inside training folds only, and every arm received the same candidate variables. Artificial intelligence was used in two further roles: to appraise 14 published machine learning studies of this outcome against the same reporting items, and to produce an adversarial peer review of this manuscript, every claim of which was verified by re-analysis. Results. Extubation failure occurred in 71 patients (8.9%). The published points score reproduced exactly, separating four risk strata with observed failure rates of 4.4%, 7.7%, 11.6% and 23.0%, but the score distributions of patients who failed and succeeded overlap almost completely. Given the same 12 candidate variables, random forest reached 0.630 and gradient boosting 0.627 against 0.655 for ordinary logistic regression, a spread of 0.028 in a design that cannot detect less than 0.068. Given all 42 variables, tuning, assembling and class rebalancing, no strategy exceeded the original score; SMOTE oversampling left discrimination unchanged while inflating predicted risk by 75% and driving net benefit below the treat-no strategy. Among the 14 published studies, 12 displayed an explainability figure, 7 reported calibration, 4 had any external validation, 3 reported net benefit, 3 shared code and none used repeated or nested cross validation. Conclusions. Artificial intelligence excels at extracting patterns from existing datasets, but it cannot generate information that is not there. In this cohort, neither advanced MLM nor extensive algorithmic tuning outperformed a parsimonious model based on clinical judgment.
Language: English
Field: Medicine
Open Access status: green
OpenAlex ID: W7236713132
Cites: 45
Cited by:
Related to: 10