Emotion Recognition Through Multimodal Bottleneck Transformers: The Integration of Speech and Facial Data to Improve Precision
Year: 2025
Type: article
Abstract: Emotion recognition is pivotal for enhancing human-computer interaction, yet accurately identifying emotions from multimodal inputs-such as facial expressions and speech-remains a substantial challenge.Traditional methods often analyze these modalities in isolation, potentially overlooking critical emotional cues that manifest when these data sources are integrated.This work introduces a novel approach through the development of Multimodal Bottleneck Transformers, a unique architectural innovation designed specifically for emotion recognition.This model is distinct from previous transformer applications in its use of a specialized bottleneck attention mechanism, which selectively narrows down the focus of attention to the most informative features of both facial and speech data.This selective attention process efficiently fuses the modalities, significantly enhancing the accuracy and robustness of emotion recognition across diverse and complex environments.Leveraging this tailored integration technique, our proposed model demonstrates superior performance, as evidenced by extensive validation on the IndEmo dataset.This dataset, encompassing a wide array of emotional expressions across 13,000 samples, reveals that our model achieves an accuracy of up to 99.2%, markedly surpassing traditional single-modality models.The results confirm the efficacy of our approach in synthesizing facial and speech data to elevate emotion recognition systems, suggesting profound implications for advancements in interactive AI and psychological analysis.
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: hybrid
OpenAlex ID: W4408495133