Get started
Work

Multimodal contrastive learning for optimized integration of ECG signals and clinical reports for arrhythmia classification

Year: 2025
Type: article
Abstract: Early and accurate detection of cardiac arrhythmias is essential for timely diagnosis and effective treatment. Traditional ECG-based classification models primarily rely on signal processing techniques, often neglecting the valuable contextual information contained in clinical notes. ‎This study introduces a novel multimodal contrastive learning framework that integrates ECG signals and clinical notes for arrhythmia clas-‎sification. This approach is not previously explored in ECG-based diagnostics. Unlike existing methods that process ECG and clinical text ‎separately, our framework employs contrastive learning for early feature alignment, ensuring meaningful representation of both modalities. ‎Additionally, we propose an alternative alignment strategy that enables ECG-text integration without requiring direct patient-level mapping, ‎addressing a major challenge in real-world healthcare applications. A neural network extracts temporal features from ECG signals, while a ‎BERT-based model processes textual clinical data. Contrastive learning is employed to align multimodal representations, improving the ‎model’s ability to differentiate between normal and abnormal arrhythmias. Experimental results demonstrate 97.3% accuracy and an AUC-‎ROC of 0.98, significantly outperforming unimodal approaches. The proposed method enhances model generalization, robustness, and ‎interpretability, making it suitable for real-world clinical applications. This study contributes to the development of AI-driven diagnostic ‎tools that can integrate multimodal patient data, improving the reliability of automated arrhythmia detection in healthcare settings‎.
DOI: 10.14419/x1f6be61
Language: English
Field: Medicine
Open Access status: diamond
OpenAlex ID: W4410826129
FWCI:
Cites:
Cited by:
Related to: 10