A novel multimodal approach for emotion recognition using Text, Speech, and Facial Expression data
Year: 2024
Type: article
Abstract: Multimodal emotion recognition (MER) is essential for improving human-computer interaction by allowing systems to comprehend and react to human emotions. This research introduces an innovative method for Multimodal Emotion Recognition (MER) by combining three modalities—text, speech, and facial expression—through a sophisticated framework that employs modality-specific feature selection, independent classification, and learnable attention-based feature fusion. The suggested approach tackles the difficulties of aligning and integrating features from several modalities while maintaining robustness and interpretability. The model adeptly adjusts to fluctuating emotional displays by utilizing the strengths of each modality. The research illustrates the efficacy of the suggested methodology via comprehensive trials, revealing substantial enhancements in classification precision and resilience relative to conventional unimodal and rudimentary multimodal systems. This research advances the creation of more empathic and contextually cognizant human-computer interaction systems, with potential applications in virtual assistants, mental health monitoring, and customer service.
Source: Cuestiones de Fisioterapia
DOI: 10.48047/pgff9y84
Authors: Nikita Joshi, Rakesh Kumar Khare
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: hybrid
OpenAlex ID: W4412685633