Refining sleep staging accuracy: transfer learning coupled with scorability models
Year: 2024
Type: article
Abstract: STUDY OBJECTIVES: This study aimed to (1) improve sleep staging accuracy through transfer learning (TL), to achieve or exceed human inter-expert agreement and (2) introduce a scorability model to assess the quality and trustworthiness of automated sleep staging. METHODS: A deep neural network (base model) was trained on a large multi-site polysomnography (PSG) dataset from the United States. TL was used to calibrate the model to a reduced montage and limited samples from the Korean Genome and Epidemiology Study (KoGES) dataset. Model performance was compared to inter-expert reliability among three human experts. A scorability assessment was developed to predict the agreement between the model and human experts. RESULTS: Initial sleep staging by the base model showed lower agreement with experts (κ = 0.55) compared to the inter-expert agreement (κ = 0.62). Calibration with 324 randomly sampled training cases matched expert agreement levels. Further targeted sampling improved performance, with models exceeding inter-expert agreement (κ = 0.70). The scorability assessment, combining biosignal quality and model confidence features, predicted model-expert agreement moderately well (R² = 0.42). Recordings with higher scorability scores demonstrated greater model-expert agreement than inter-expert agreement. Even with lower scorability scores, model performance was comparable to inter-expert agreement. CONCLUSIONS: Fine-tuning a pretrained neural network through targeted TL significantly enhances sleep staging performance for an atypical montage, achieving and surpassing human expert agreement levels. The introduction of a scorability assessment provides a robust measure of reliability, ensuring quality control and enhancing the practical application of the system before deployment. This approach marks an important advancement in automated sleep analysis, demonstrating the potential for AI to exceed human performance in clinical settings.
Source: SLEEP
Language: English
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Open Access status: green
Funders: National Science Foundation, Ministry of Education, Korea Disease Control and Prevention Agency, Ministry of Health and Welfare, Korea Health Industry Development Institute, National Institutes of Health, National Institute on Aging, National Heart, Lung, and Blood Institute, National Institute of Neurological Disorders and Stroke
Awards: G122882294, Multimodal Network Connectivity Architecture (MOCA) of the Brain and its Role in the Recovery of Consciousness in Comatose Cardiac Arrest Patients, SCH: INT: Collaborative Research: DeepSense: Interpretable Deep Learning for Zero-effort Phenotype Sensing and Its Application to Sleep Medicine, G524119356, G5277991875, Establishing a Brain Health Index, G580858672, G5843205994, G6315715174, Investigation of Sleep, G6781386558, G7403249553, G7637461272, G808739799, Integrative Motor Activity Biomarker for the Risk of Alzheimer's Risk, G9000066320, G9000081834, G9000137206, Data-Driven Sleep Biomarkers of Brain Health, Heart Health, and Mortality, Prospective Validation of Neurophysiologic Outcome Prediction in Acute Brain Injury
OpenAlex ID: W4402078496