Multimodal Symptom-to-ICD-11 Group Classification with External Validation Across Emergency Departments (Preprint)
Year: 2026
Type: preprint
Abstract: BACKGROUND Emergency department (ED) decisions are often made under severe time pressure and clinical uncertainty, yet symptom-based decision-support systems with external validation remain limited, particularly for low-resource clinical languages. OBJECTIVE To develop and externally validate a multimodal model for classifying ED encounters into ICD-11 disease groups using early-available Uzbek-language complaint text and structured triage information. METHODS We conducted a retrospective multicenter prediction study of 3,360 ED encounters, including 2,348 encounters in a development cohort and 1,012 in an independent external validation cohort. Uzbek complaint narratives were processed through a UZ-EDBench-derived symptom extraction and normalization pipeline and combined with structured symptom descriptors, demographics, and intake vital signs in a multimodal fusion model. Performance was compared with multinomial logistic regression, gradient boosting, and a text-only transformer. The primary endpoint was macro-F1; secondary endpoints included top-3 accuracy, macro-AUROC, Brier score, expected calibration error (ECE), and subgroup robustness. RESULTS The Uzbek symptom extraction layer achieved a token micro-F1 of 0.944 and an entity exact-match F1 of 0.861. The multimodal model outperformed all baselines on both internal temporal testing and external validation, achieving macro-F1 values of 0.691 and 0.654, top-3 accuracies of 0.901 and 0.868, and macro-AUROC values of 0.914 and 0.889, respectively. Compared with the best structured baseline, macro-F1 improved by 0.043 internally and 0.056 externally. Temperature scaling reduced ECE from 0.049 to 0.024 internally and from 0.056 to 0.031 externally. CONCLUSIONS Integrating Uzbek complaint text with structured triage variables enabled clinically meaningful ICD-11 disease-group stratification and retained performance under cross-site validation. These findings support calibration-aware, language-aware ED decision support for early stratification and differential prioritization rather than autonomous diagnosis.
Authors: Botir Elov, Ruhillo Alaev
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7154743628