MSA-GCN: A Graph Convolutional Network Approach for Internal Feature Fusion in Multimodal Sentiment Analysis
Year: 2025
Type: conference-paper
Abstract: Multimodal Sentiment Analysis (MSA) improves the accuracy of sentiment prediction by fusing three modalities: text, audio, and visual. However, we found that there are two key limitation problems prevalent in the existing methods: on the one hand, the existing techniques overly focus on the realization of inter-modal interactions but neglect the mining of spatiotemporal features within individual modalities; on the other hand, the traditional fusion strategies are not refined enough to effectively model fine-grained spatiotemporal dependency pathways. Therefore, to solve the above two problems, we propose a multimodal sentiment analysis framework based on graph convolution network (GCN), MSA-GCN(Multimodal Spatiotemporal Attention Graph Convolutional Network), which strengthens the modal internal feature representation through the hierarchical graph convolution mechanism, and innovatively constructs an independent spatiotemporal graph model, so that cross-modal fusion can be realized while fully exploiting the spatial and temporal features of single modality. The experimental results show that MSA-GCN significantly outperforms other baseline algorithms in several metrics, especially in the long-time sequence emotion recognition task.
Authors: Ao Mei, Jiahao Mei, Tianyi Hao, Mukun Rao, Ziyang Zhang, Dengshi Li
Institution: Jianghan University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4412789805