Year: 2026
Type: conference-paper
Abstract: The advancement of Multimodal Emotion Recognition (MER) in Chinese is significantly hindered by the scarcity of high-quality, spontaneous dialogue datasets compared to their English counterparts.In this work, we introduce EmotionTalk, the first interactive Chinese multimodal dataset designed to capture the nuance of authentic emotional interplay.Collected from 19 professional actors, the dataset spans 23.6 hours of dyadic conversations across diverse scenarios.A key contribution of EmotionTalk is its multi-grained annotation system, which integrates standard categorical and dimensional labels with fine-grained emotional speaking style captions, enabling research into interpretable emotion analysis.We establish comprehensive benchmarks for emotion recognition and captioning tasks, verifying the dataset's effectiveness and the necessity of multimodal fusion.EmotionTalk serves as a critical resource for bridging the gap in non-English affective computing and is publicly released for the research community.
Authors: Haoqin Sun, Jinghua Zhao, Xuechen Wang, Shiwan Zhao, Jiaming Zhou, Hui Wang, Xi Yang, Yequan Wang, Yonghua Lin
Institutions: Nankai University, Beijing Academy of Artificial Intelligence
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: hybrid
Funder: National Science Foundation
OpenAlex ID: W4416608640