A Two-Stage Framework for Multimodal Aspect-Based Sentiment Analysis Using Cross-modal Translation
Year: 2024
Type: conference-paper
Abstract: In recent years, there has been a growing interest in the task of multimodal aspect-based sentiment analysis (MABSA). The goal of this task is to combine visual and textual information to predict the sentiment polarity of the target mentioned in the text. Existing research mainly focuses on the fusion of multimodal features using attention methods. However, these methods face challenges due to the representation gap between different modalities, and the fusion of different modalities increases the risk of introducing noise in modality interactions. In this paper, we propose a novel two-stage framework with cross-modal translation (TSFCMT) for MABSA. To address the issue of semantic representation gap, we leverage image captions as a bridge between visual and textual modalities. Additionally, to avoid introducing noise in modality interactions during cross-modal fusion, we propose a two-stage module that separately utilizes textual and visual cues for aspect sentiment classification. Furthermore, we introduce an aspect-based decision module to comprehensively integrate aspect sentiment polarities from different modalities. Experimental results on two publicly available multimodal aspect-based sentiment analysis datasets demonstrate that our framework outperforms strong baseline models.
Authors: Wencong Wang, Junjie Chen, Hao Fan
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Award: 基于图神经网络的短文本主题情感分析方法的研究
OpenAlex ID: W4409832937