Year: 2025
Type: preprint
Abstract: Vision-Language Models (VLMs) have emerged as powerful tools for multimodal understanding, demonstrating remarkable capabilities in combining visual and textual information. This paper presents a comprehensive technical review of fifteen state-of-the-art VLMs under 20B parameters, including recent architectures like Llama 3.2 Vision, Qwen2-VL, and Phi-3.5 Vision. We analyze the architectural evolution of these models, from early encoder-decoder designs to modern developments in Mixture-of-Experts (MoE) and progressive learning approaches. The review examines key technical innovations in cross-modal attention mechanisms, visual feature integration strategies, and parameter-efficient designs. We provide detailed analysis of architectural components such as vision encoders, attention mechanisms, and integration strategies that define each model's approach to vision-language understanding. Through this systematic review, we identify emerging trends in VLM architecture design, including the rise of lightweight yet effective models and the growing adoption of MoE-based approaches. This comprehensive analysis offers valuable insights for researchers and practitioners in understanding the current landscape of vision-language modeling and its architectural directions.
Source: HAL
Authors: Mukherjee, Arka, Paul, Anuruddha, Halder, Aritro
Institution: KIIT University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4406791822
