Work

CLAIRE: a CNN–LSTM framework with adaptive and iterative refinement for multimodal emotion recognition

Year: 2026

Type: article

Abstract: Recent breakthroughs in Deep Learning and bio signal processing have encouraged researchers to model human–computer interaction with greater accuracy. Emotion recognition is critical in facilitating the understanding and reaction of machines to the affective states of human beings. Many emotion recognition approaches have been suggested to identify emotions based on unimodal features. However, the use of unimodal features alone has resulted in more misclassification errors due to the presence of overlapping patterns among similar emotions. Although determining emotions in multimodal signal provides a higher recognition rate, it remains a challenging task. To address this challenge, this study proposes CLAIRE (CNN–LSTM with Adaptive and Iterative Refinement), a representation-level multimodal emotion recognition framework that independently learns modality-specific features from speech, EEG, and ECG using dedicated CNN–LSTM branches. Instead of performing direct sample-level fusion, the learned latent representations are projected into a shared embedding space and integrated through an adaptive fusion strategy for robust emotion classification. Furthermore, an Adaptive Grid Search Refinement (AGSR) algorithm is introduced to optimize the network hyperparameters efficiently while reducing computational overhead. Experimental evaluation demonstrates that the proposed framework achieves an overall emotion recognition accuracy of 97.2%, outperforming the considered baseline methods while exhibiting improved robustness and computational efficiency. These results demonstrate the effectiveness of representation-level fusion for heterogeneous emotion datasets and highlight the potential of the proposed framework for affective computing applications.

Source: Scientific Reports

DOI: 10.1038/s41598-026-67350-y

Authors: S. Radhika, Prasanth Aruchamy, Kranthi Kumar Singamaneni

Institutions: Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology, Symbiosis International University

Language: English

Topic: Emotion and Mood Recognition

Subfield: Experimental and Cognitive Psychology

Field: Psychology

Domain: Social Sciences

Open Access status: gold

Estimated APC paid: $2,690

Funder: Symbiosis International University