Explainable machine learning for leakage-aware risk stratification of electrocardiograph verification records
Year: 2026
Type: preprint
Abstract: Metrological verification of electrocardiographs (ECGs) performance and safety generates structured technical and measurement data that may support the identification of devices requiring additional expert review. Machine-learning models may assist verification workflows by devices at elevated risk of failure. This study investigates the predictive value and limitations of routinely collected metrological verification data for identifying electrocardiographs at increased risk of verification failure. A retrospective analysis of 3,956 electrocardiograph verification records, including 3,870 passing and 86 failing records, was conducted. Technical and measurement variables available before the final verification decision were selected, while variables associated with potential data leakage were excluded. Three machine-learning algorithms, LightGBM, Balanced Random Forest, and SMOTE Random Forest, were compared using repeated stratified five-fold cross-validation, with PR-AUC as the primary model selection criterion. The selected model was further evaluated using out-of-fold predictions, bootstrap confidence intervals, calibration analysis, feature importance, and group-based validation to assess performance on previously unseen manufacturers and device models. Among the evaluated algorithms, LightGBM achieved the highest precision-recall performance, demonstrating substantial improvement over the baseline failure prevalence and revealing meaningful predictive information in routinely collected metrological data. Explainability analyses identified measurement-related characteristics and device-specific information as important predictors, while ablation analysis demonstrated that predictive discrimination was retained even after removing device identity. The study further demonstrated the value of combining leakage-controlled modelling, explainability, and robustness assessment to critically evaluate machine-learning performance in real-world medical device verification. Nevertheless, limited precision, poor probability calibration, and reduced performance on unseen manufacturers and device models highlighted important limitations, while decision curve analysis provided insufficient evidence of consistent operational benefit. This study demonstrates the potential of routinely collected metrological verification data to support AI-driven risk stratification, contributing to the broader transition toward data-driven medical device management advocated by the World Health Organization. While the findings establish a basis for further development of intelligent quality assurance systems, limitations in precision, calibration, and generalizability highlight the need for independent validation and demonstrated operational benefit before implementation in routine verification practice.
Source: Research Square
Institutions: University of Zenica, Clinical Center University of Sarajevo
Language: English
Subfield: Medical Laboratory Technology
Field: Health Professions
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7236626342
