A hierarchical attention-based multimodal fusion framework for predicting the progression of Alzheimer’s disease
Year: 2023
Type: article
Abstract: Early detection and treatment can slow the progression of Alzheimer's Disease (AD), one of the most common neurodegenerative diseases. Recent studies have demonstrated the value of multimodal fusion in early AD detection. However, most approaches to this have failed to consider data modality domains, their relationships, and variations in their relative importance. To address these challenges, we propose a Hierarchical Attention-Based Multimodal Fusion framework (HAMF) that utilizes imaging, genetic and clinical data for early AD detection. In the HAMF model, attention mechanisms are utilized to learn the appropriate weights for each modality and to understand the interaction between modalities through hierarchical attention. HAMF performs better than state-of-the-art methods, achieving an accuracy of 87.2% and an AUC of 0.913, which are superior to unimodal models. By comparing the results of different unimodal and multimodal models, we find that multimodal fusion can improve model performance more than unimodal models and clinical data is the most important modality. Our ablation experiment confirmed the effectiveness of HAMF. Finally, we used SHapley Additive exPlanations (SHAP) to improve the model's interpretability. We provide the model as a guide for future research in the field, and as a framework for generating actional advice and decision support system for clinical practitioners.
Language: English
Subfield: Psychiatry and Mental health
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
Estimated APC paid: $2,420
Funders: National Natural Science Foundation of China, Natural Science Foundation of Hubei Province, National Office for Philosophy and Social Sciences, Medical Science and Technology Foundation of Guangdong Province, Young Scientists Fund, National Key Research and Development Program of China, National Social Science Fund of China, Basic and Applied Basic Research Foundation of Guangdong Province
OpenAlex ID: W4388473567
