Year: 2026
Type: conference-paper
Abstract: Multimodal models struggle with idiomatic expressions due to their non-compositional meanings, a challenge amplified in multilingual settings.We introduced PolyFrame, our system for the MWE-2026 AdMIRe 2 shared task on multimodal idiom disambiguation, featuring a unified pipeline for both image+text ranking (Subtask A) and text-only caption ranking (Subtask B).All model variants retain frozen CLIP-style vision-language encoders and the multilingual BGE M3 encoder, training only lightweight modules: a logistic regression and LLM-based sentence-type predictor, idiom synonym substitution, distractor-aware scoring, and Borda rank fusion.Starting from a CLIP baseline (26.7% Top-1 on English dev, 6.7% on English test), adding idiom-aware paraphrasing and explicit sentence-type classification increased performance to 60.0% Top-1 on English, and 60.0% Top-1 (0.822 NDCG@5) in zero-shot transfer to Portuguese.On the multilingual blind test, our systems achieved average Top-1/NDCG scores of 0.35/0.73for Subtask A and 0.32/0.71for Subtask B across 15 languages.Ablation results highlight idiomaware rewriting as the main contributor to performance, while sentence-type prediction and multimodal fusion enhance robustness.These findings suggest that effective idiom disambiguation is feasible without fine-tuning large multimodal encoders.
Author: Nina Hosseini-Kivanani
Institution: University of Luxembourg
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7140121852