Augmenting Vision-Language Retrieval: The Role of Multimodal LLMs as Synthetic Data Generators
Year: 2025
Type: conference-paper
Abstract: Multimodal Large Language Models (MLLMs) connect and interpret different data types, making them suitable for various vision-language tasks. Despite the rapid advancements in MLLMs, their effectiveness for specialized cross-modal retrieval tasks remains underexplored. A challenging example is art retrieval, where the task is to find visually and conceptually relevant artwork corresponding to a textual description. This paper investigates the effects of fine-tuning cross-modal retrieval models using both human-annotated and MLLM-generated captions for artistic paintings. To this end, two cross-modal retrieval models, Long-CLIP and BLIP, are studied. Experimental results show that models fine-tuned on MLLM-generated captions achieve search effectiveness comparable to those fine-tuned on human-annotated captions.
Authors: Aidan Bell, James Gore, Behrooz Mansouri
Institution: University of Southern Maine
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4412376973