Work

HiRoC: selective history routing and disagreement-aware calibration for multimodal conversational emotion recognition

Year: 2026

Type: article

Abstract: Multimodal Emotion Recognition in Conversation (MERC) aims to infer speakers' affective states from heterogeneous cues within evolving dialogue contexts. Although graph-based methods have shown strong potential in modeling conversational structures, they often introduce history through dense or predefined connections and obtain unified multimodal representations before prediction. This paradigm may propagate irrelevant contextual information, overlook the different roles of same-speaker and cross-speaker dependencies, and underexploit useful corrective cues carried by inconsistent modalities. To address these limitations, we propose HiRoC, a multimodal emotion recognition framework that selectively routes emotion-relevant history and calibrates predictions using cross-modal disagreement. Specifically, a Historical Relevance Filter (HRF) scores how relevant each past utterance is to the current one and suppresses weakly related context before propagation. Guided by the resulting dependency prior, a Speaker-Typed Routing Graph (STRG) keeps intra- and inter-speaker dependencies apart and propagates the retained history along sparse, temporally directed paths. In parallel, the filtered modality-specific representations feed a hypergraph backbone for high-order utterance- and modality-level interactions, which the routing branch complements with speaker-aware temporal dependencies. A Cross-Modal Residual Correction (CMRC) module then uses modal disagreement to adjust the main prediction at the decision level, so that conflicting cues are not averaged away. On IEMOCAP and MELD, HiRoC outperforms representative state-of-the-art methods.

Source: Journal of King Saud University - Computer and Information Sciences

DOI: 10.1007/s44443-026-01163-x