AURA: Alignment-Uniformity Representation Approach in Multi-modal Learning
Year: 2026
Type: preprint
Abstract: Multi-modal contrastive learning, exemplified by CLIP-style vision-language models, has driven significant progress in cross-modal understanding. However, the effectiveness of these models is fundamentally constrained by the design of the contrastive loss function, which couples the objectives of alignment and uniformity within a single softmax formulation. The widely-used InfoNCE loss conflates alignment with uniformity and fails to independently control intra-modal versus cross-modal repulsion, leading to an uncontrolled modality gap and suboptimal representation geometry. This paper introduces AURA (Alignment-Uniformity Representation Approach), a novel loss function that explicitly decomposes uniformity into four independent components: image–image repulsion L_ii, text–text repulsion L_tt, image–text repulsion L_ic, and text–image repulsion L_tc. We conduct a comprehensive evaluation across 12 model variants (4 vision encoders × 3 text encoders) with 3 loss functions, measuring 6 metrics on 2 datasets (COCO, XM3600). AURA achieves the lowest Uniformity on 12/12 configurations and reduces Modality Gap by up to ∼55–60% relative to NT-Xent, while maintaining competitive or superior recall. This comes at the cost of higher Alignment scores in some configurations; we provide theoretical analysis via gradient dynamics, Riesz potential, and mutual information bounds to explain this trade off and the geometric properties of the learned embedding space.
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7171078221
Cites:
Cited by:
Related to: