VLCAP at ImageEval 2025 Shared Task: Multimodal Arabic Captioning with Interpretable Visual Concept Integration
Year: 2025
Type: conference-paper
Abstract: We present VLCAP, an Arabic image captioning framework that integrates CLIP-based visual label retrieval with multimodal text generation.Rather than relying solely on end-toend captioning, VLCAP grounds generation in interpretable Arabic visual concepts extracted with three multilingual encoders, mCLIP, Ara-CLIP, and Jina V4, each evaluated separately for label retrieval.A hybrid vocabulary is built from training captions and enriched with about 21K general domain labels translated from the Visual Genome dataset, covering objects, attributes, and scenes.The top-k retrieved labels are transformed into fluent Arabic prompts and passed along with the original image to vision-language models.In the second stage, we tested Qwen-VL and Gemini Pro Vision for caption generation, resulting in six encoder-decoder configurations.The results show that mCLIP + Gemini Pro Vision achieved the best BLEU-1 (5.34%) and cosine similarity (60.01%), while AraCLIP + Qwen-VL obtained the highest LLM-judge score (36.33%).This interpretable pipeline enables culturally coherent and contextually accurate Arabic captions.
Authors: Passant Elchafei, Amany Fashwan
Institutions: Universität Ulm, Alexandria University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416037776