Work

FAST-MEL: A Fast, Accurate, and Storage Efficient Solution for Multimodal Entity Linking

Year: 2026

Type: conference-paper

Abstract: Multimodal entity linking (MEL) is the task that consists of matching textual and visual mentions of entities in unstructured data to their corresponding entities in a knowledge base (KB). To be effective in large-scale practical settings, MEL systems must meet three objectives: high linking accuracy, computational efficiency, and storage efficiency, i.e., a compact yet efficient index of the KB. In this paper, we highlight that state-of-the-art systems fail to simultaneously satisfy these 3 requirements. To meet this three-fold objective, we propose FAST-MEL, a lightweight encoder-based MEL solution that relies on a novel and compact fixed-size vectorized representation of both the textual and visual information of each entity or mention. It matches the accuracy of the best systems but performs three orders of magnitude faster. It also consumes one order of magnitude less storage than the fastest systems.

Source: International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

DOI: 10.1145/3805712.3809860

Authors: Thomas Derrien, Laurent Amsaleg, Pascale Sébillot

Institutions: Centre National de la Recherche Scientifique, Institut national de recherche en sciences et technologies du numérique, Institut de Recherche en Informatique et Systèmes Aléatoires, Institut National des Sciences Appliquées de Rennes, Université de Rennes

Language: English

Topic: Multimodal Machine Learning Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: Agence Nationale de la Recherche

Award: discoverAbility and investiGation in spAtial iconograPhic hEritage