Year: 2026
Type: article
Abstract: The task of scene parsing has historically relied on unimodal architectures that process visual data in isolation, categorizing pixels based on closed-set vocabularies. However, the emergence of multimodal foundation models has fundamentally altered the landscape of computer vision and natural language processing. By projecting visual and textual data into a shared embedding space, these models enable open-vocabulary comprehension and zero-shot generalization. This paper investigates the integration of multimodal foundation models for the complex task of scene parsing, emphasizing the critical challenge of semantic alignment between dense pixel representations and granular textual descriptions. We propose a novel framework that bridges the gap between image-level linguistic embeddings and pixel-level semantic requirements. Our approach leverages a hierarchical vision transformer coupled with a cross-modal alignment module that enforces localized feature congruence through contrastive learning mechanisms. Through extensive empirical evaluation on benchmark datasets, we demonstrate that incorporating rich language priors significantly enhances the disambiguation of visually similar regions and improves robustness against diverse environmental conditions. Furthermore, our analysis dissects the specific mechanisms by which foundation models distribute attention across modalities, revealing that dense semantic alignment accelerates convergence and yields highly interpretable parsing results. The findings establish a robust baseline for future explorations into open-vocabulary scene understanding and multimodal feature fusion, providing insights into the architectural optimizations necessary for deploying these models in resource-constrained environments.
Author: Amelia Laurent
Institution: University of Oxford
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7220247336