ProMahaVQA: Enhancing Zero-Shot Visual Question Answering with Prototype Learning and Contrastive Loss
Year: 2025
Type: article
Abstract: 视觉问答(VQA)是一项复杂的人工智能任务，要求模型理解图像内容与自然语言问题，实现跨模态语义融合。然而，现有方法在处理视觉与语言深度交互方面存在明显不足，尤其在零样本场景中泛化能力有限。为此，本文提出ProMahaVQA模型，引入跨模态原型矩阵、原型查询机制与基于马氏距离的多标签对比损失，有效提升了特征判别能力与模型鲁棒性。模型首次将原型学习机制应用于零样本VQA任务，并通过记忆矩阵支持对未见答案的识别。实验结果表明，ProMahaVQA在F-VQA、TZSL和GZSL等设置下均显著优于现有方法，展现出卓越的泛化性能与跨模态推理能力。Visual Question Answering (VQA) is a challenging artificial intelligence task that requires models to comprehend image content and natural language questions through cross-modal semantic integration. However, existing methods often struggle with deep visual-language interactions, particularly in zero-shot scenarios where generalization is limited. To address these challenges, we propose ProMahaVQA, a novel model that incorporates a cross-modal prototype matrix, a prototype query mechanism, and a Mahalanobis distance-based multi-label contrastive loss. These innovations significantly enhance feature discrimination and model robustness. Notably, this work is the first to integrate prototype learning into zero-shot VQA, enabling the model to recognize unseen answers via a memory matrix. Experimental results on F-VQA, TZSL, and GZSL benchmarks demonstrate that ProMahaVQA substantially outperforms existing approaches, exhibiting superior generalization and cross-modal reasoning capabilities.
Source: Statistics and Applications
Author: 婧昕 闫
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4411462204