Enhancing Human–Robot Interaction: Development of Multimodal Robotic Assistant for User Emotion Recognition
Year: 2024
Type: article
Abstract: This paper presents a study on enhancing human–robot interaction (HRI) through multimodal emotional recognition within social robotics. Using the humanoid robot Pepper as a testbed, we integrate visual, auditory, and textual analysis to improve emotion recognition accuracy and contextual understanding. The proposed framework combines pretrained neural networks with fine-tuning techniques tailored to specific users, demonstrating that high accuracy in emotion recognition can be achieved by adapting the models to the individual emotional expressions of each user. This approach addresses the inherent variability in emotional expression across individuals, making it feasible to deploy personalized emotion recognition systems. Our experiments validate the effectiveness of this methodology, achieving high precision in multimodal emotion recognition through fine-tuning, while maintaining adaptability in real-world scenarios. These enhancements significantly improve Pepper’s interactive and empathetic capabilities, allowing it to engage more naturally with users in assistive, educational, and healthcare settings. This study not only advances the field of HRI but also provides a reproducible framework for integrating multimodal emotion recognition into commercial humanoid robots, bridging the gap between research prototypes and practical applications.
Source: Applied Sciences
DOI: 10.3390/app142411914
Authors: Sergio Garcia, Francisco Gomez-Donoso, Miguel Cazorla
Institution: University of Alicante
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,728
OpenAlex ID: W4405565614