Work

Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysis

Year: 2025

Type: conference-paper

Abstract: Multimodal Sentiment Analysis (MSA) integrates diverse modalities—text, audio, and video—to comprehensively analyze and understand individuals’ emotional states. However, the real-world prevalence of incomplete data poses significant challenges to MSA, mainly due to the randomness of modality missing. Moreover, the heterogeneity issue in multimodal data has yet to be effectively addressed. To tackle these challenges, we introduce the Modality-Invariant Bidirectional Temporal Representation Distillation Network (MITR-DNet) for Missing Multimodal Sentiment Analysis. MITR-DNet employs a distillation approach, wherein a complete modality teacher model guides a missing modality student model, ensuring robustness in the presence of modality missing. Simultaneously, we developed the Modality-Invariant Bidirectional Temporal Representation Learning Module (MIB-TRL) to mitigate heterogeneity.

Source: IEEE International Conference on Acoustics Speech and Signal Processing

DOI: 10.1109/icassp49660.2025.10888606

Authors: Xincheng Wang, Liejun Wang, Yinfeng Yu, Xinxin Jiao

Institution: Xinjiang University

Language: English

Topic: Sentiment Analysis and Opinion Mining

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Natural Science Foundation of China, Technology Development

Award: G8631799244