FusionGuard: A Multimodal Adversarially Aware Classifier for Robust Image-Text Classification
Year: 2025
Type: article
Abstract: From social media content moderation to medical image diagnosis and multimedia retrieval, multimodal classification models that integrate textual and visual information are increasingly becoming the center of interest.Despite their improved performance over unimodal systems, these models are often vulnerable to adversarial attacks that exploit modality-specific weaknesses that lead to misclassification and unreliable outcomes.Moreover, real-world datasets commonly suffer from class imbalance which further degrades model generalization.To address these challenges, the researchers developed FusionGuard, a novel multimodal classification framework that combines the complementary strengths of TinyBERT for text encoding and EfficientNet for image feature extraction within a hybrid fusion architecture.Furthermore, they incorporated adversarial training to enhance robustness against adversarial attacks.Additionally, the Synthetic Minority Oversampling Technique (SMOTE) was used to mitigate class imbalance, alongside focal loss optimization to focus learning on difficult examples and reduce bias.The team obtained 80% accuracy score, 79.71% macro precision and 79.21% macro F1 score against FGSM attacks respectively.The results obtained establish their model as a reliable and fair solution for robust multimodal learning in security-critical applications.
Institution: University of Buea
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4413888143