Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspective
Year: 2026
Type: conference-paper
Abstract: Multimodal Emotion Recognition (MER) systems often suffer from missing modalities in real-world scenarios.Existing methods usually generate, align, or distill missing modalities as a whole, overlooking the heterogeneous nature of the information carried by each modality.Such holistic treatment mixes inferable shared semantics with uncertain modality-specific details, yielding unstable representations and degrading robustness.To address this issue, we propose the Primitive Memory Distillation (PriMD) framework.Unlike existing methods, PriMD takes an intra-modal perspective and focuses on how different types of information within a modality differ in recoverability.PriMD first disentangles cross-modal shared semantics from modality-specific representations, and then discretizes the latter into learnable semantic primitives to construct modality-specific memory banks.When modalities are missing, PriMD is a teacher-student framework that the student model uses the shared semantics of available modalities as queries to dynamically retrieve primitives.It compensates for missing modality-specific information within a constrained memory space and aligns with the teacher model.Extensive experiments on IEMOCAP, CMU-MOSI, and CMU-MOSEI demonstrate that PriMD achieves state-of-the-art performance and consistently stronger robustness across a wide range of missing-modality settings, while mitigating the instability caused by holistic feature inference.Our code and project website are available at https
Institutions: Jiangsu University, Griffith University, National University of Singapore, UNSW Sydney, Fujian University of Technology, Xi’an Jiaotong-Liverpool University, German Research Centre for Artificial Intelligence, Tsinghua University, Imperial College London, Technical University of Munich, Queen Mary University of London
Language: English
Topic: Emotion and Mood Recognition
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7204952672