Integrating Semantic, Sentiment, and Object-Level Cues for Multimodal Video-Based Fake News Detection
Year: 2025
Type: conference-paper
Abstract: The rapid rise of short-form video platforms such as TikTok has significantly intensified the dissemination of fake news in complex multimodal formats. Traditional unimodal detection methods are inadequate for handling rich and heterogeneous content that integrates textual, auditory, and visual information. Although multimodal approaches offer greater potential, they still face fundamental challenges in effectively extracting, aligning, and integrating features from disparate modalities into a coherent and informative representation. To address these challenges, this study introduces a sentiment-aware semantic fusion framework, SASFEND, that jointly incorporates semantic, sentiment, and object-level features while preserving the distinct informational contributions of each modality. Specifically, we extract both sentiment and semantic representations from the textual, auditory, and visual modalities, and then explore the impact of various fusion strategies on fake news detection performance. Experimental results demonstrate that the proposed model outperforms existing baselines, achieving an accuracy of 79.60% and a macro F1-score of 78.19%. Notably, the inclusion of object-level visual features substantially improves the model's capacity to identify misinformation. Furthermore, comparisons across fusion strategies reveal that simple and interpretable fusion techniques can yield strong performance without resorting to overly complex architectures, highlighting their scalability and practical applicability in modern media environments.
Authors: Ruby Ham, Yuchen Zhang, Zeyu Fu
Institutions: University of Exeter, University of Essex
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W7090279574