Year: 2026
Type: conference-paper
Abstract: In multimodal information retrieval (MMIR), candidates relevant to an input query need to be retrieved from a database, where the query and database items span different modalities.As real-world databases evolve, repeatedly annotating and indexing data and re-optimizing domain-specific models across modalities is impractical.We present Multi-Score, a finetuning-free, two-stage MMIR approach that couples efficient candidate filtering with finegrained multimodal re-ranking.Stage-1 adopts Matryoshka representations to efficiently filter out low-relevance candidates without expensive similarity computations on full-scale representations for the entire database.Stage-2 reranks the filtered candidates by computing their fine-grained multimodal contextual representations with two scoring functions for semantic alignment using chain-of-thought prompting and question-answering.Experiments demonstrate state-of-the-art zero-shot retrieval on 12 MMIR tasks across 32 datasets while outperforming supervised methods on 23 datasets.
Authors: Sourajit Saha, Tejas Gokhale
Institution: University of Maryland, Baltimore County
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Award: G6181661201
OpenAlex ID: W7166807435