Cross-Modal Retrieval with Modality-Agnostic Embeddings Towards Universal Representation Learning
Year: 2026
Type: article
Abstract: A single vector space that accepts images, text, audio, video and sensor readings, and that supports retrieval between any pair of them, has become an explicit goal of multimodal research. Systems now exist that approach it by binding every modality to one anchor, which produces retrieval between pairs that were never trained together. This article examines what such spaces achieve and where they fail. We first analyse the alignment topologies available, showing that binding through an anchor reduces the requirement for paired data from a quadratic to a linear function of the number of modalities, and that the reduction has a measurable cost in the quality of pairings the anchor does not mediate. We then examine the geometry of the resulting space. Published analysis has shown that representations of different modalities occupy separate regions rather than mixing, a separation that persists throughout training and that has consequences for retrieval which are widely overlooked. Hubness and anisotropy compound the effect at the scale required for practical search. We compare seven systems by their binding topology and coverage, review the approximate search structures that make retrieval feasible over billions of items, and assess the claim that models trained on different modalities are converging on a shared statistical description of the world. Six research problems are identified.
Author: Rejina P V
Institution: Kannur University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7235515903
