Work

Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion

Year: 2026

Type: conference-paper

Abstract: Emotion recognition in multi-speaker conversations faces significant challenges due to speaker ambiguity and severe class imbalance.We propose a novel architecture that addresses these issues through three key innovations: (1) a speaker identification module that leverages audio-visual synchronization to accurately identify the active speaker, (2) a knowledge distillation strategy that transfers superior textual emotion understanding to audio and visual modalities, and (3) hierarchical attention fusion with composite loss functions to handle class imbalance.Comprehensive evaluations on MELD and IEMOCAP datasets demonstrate superior performance, achieving 67.75% and 72.44% weighted F1 scores respectively, with particularly notable improvements on minority emotion classes.

Source: Findings of the Association for Computational Linguistics: EACL

DOI: 10.18653/v1/2026.findings-eacl.212

Authors: Li Xiao, Kotaro Funakoshi, Manabu Okumura

Institution: Institute of Science Tokyo

Language: English

Topic: Emotion and Mood Recognition

Subfield: Experimental and Cognitive Psychology

Field: Psychology

Domain: Social Sciences

Open Access status: hybrid

Funder: Japan Society for the Promotion of Science

OpenAlex ID: W7140097045