Work

HCF-Net: Hierarchical and conflict-aware fusion for robust multimodal sentiment understanding

Year: 2026

Type: article

Abstract: Multimodal sentiment understanding in opinion videos must remain robust when linguistic, acoustic, and visual signals disagree, such as in sarcasm or deadpan delivery. Many existing fusion methods assume cross-modal complementarity and entangle agreement and disagreement within a single representation, which can degrade the robustness. We formulate multimodal fusion as a decomposition problem, treating aligned and conflicting evidence as distinct and complementary sources of information. We propose HCF-Net, a lightweight instantiation of this decomposition, for joint sentiment classification and intensity regression. Each modality is encoded using a two-tier hierarchical attention mechanism to capture local micro-cues and discourse-level dependencies efficiently, and fusion is performed via a dual-branch module that separates alignment from disagreement-preserving residual signals using supervised gating. Experiments on CMU-MOSI achieved 0.627 MAE with a Pearson correlation of 0.823, and 94.1% Acc-2 and 52.8% Acc-7. The results on CMU-MOSEI and UR-FUNNY confirmed the generalization of the model.

Source: Systems and Soft Computing

DOI: 10.1016/j.sasc.2026.200517

Authors: Papri Saha, Ahmed Mostakim Fahim, Md Tanzeem Rahat, Kamruddin Nur, Debasish Ghose

Institutions: American International University-Bangladesh, Høyskolen Kristiania

Language: English

Topic: Sentiment Analysis and Opinion Mining

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,870

Funder: Direktoratet for internasjonalisering og kvalitetsutvikling i høgare utdanning