Efficient Self-Attention and Co-Attention within the Modified Graph Matching Attention Network for Visual Question Answering
Year: 2024
Type: conference-paper
Abstract: Visual question answering is an evolving field at the intersection of computer vision and natural language processing, aiming to enable machines to understand and respond to questions about images.In this research, new self-attention and co-attention modules have been proposed.Self-attention is utilized to compute the intra-dependency among the elements, while the co-attention module computes cross-modal attention among different modalities.The research has achieved 66.30% accuracy on the validation split of the VQA 2.0 dataset when utilizing the co-attention module with the implicit stage.Additionally, the computational complexity of the proposed VQA model has been reduced by using only self-attention and co-attention, although it has resulted in a lesser accuracy than the previous model, with an accuracy of 65.74% on the validation dataset.From the research survey, it is observed that the self-attention and co-attention module have yielded good results for the 'Number' case of the testing dataset, attaining 47.84% accuracy for the test-dev and 48.15% accuracy for the test-standard, surpassing the previous Graph Matching Attention (GMA) model by 0.93% and 1.54%, respectively.However, for the 'Other' case, Coattention with the implicit stage has achieved better performance.This model has outperformed the previous Graph Matching Attention model by 0.38% and 0.2% for the test-dev and test-standard, respectively.But, the proposed model didn't perform well for the 'Yes/No' case in the testing dataset.Overall, the proposed model has outperformed the previous model on the VQA 2.0 dataset.It is hoped that this research will serve as a roadmap for researchers and practitioners interested in advancing the state-of-the-art in visual question answering.
Authors: Md. Fakhrul Islam, Md. Faysal Ahamed, Md. Rabiul Islam
Institution: Rajshahi University of Engineering and Technology
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4411097836