Conservative Proxy Prompting for Reliable Frozen Vision–Language Models with Missing Modalities
Year: 2026
Type: preprint
Abstract: Frozen vision--language models have achieved strong transferability by learning aligned image--text representations, but their reliability can degrade when one modality becomes unavailable in practical scenarios. Existing missing-modality methods often reconstruct unavailable information; however, recovered representations may contain uncertain signals and introduce misleading evidence. To address this issue, we propose Conservative Proxy Prompting (CPP), a parameter-efficient framework for adapting frozen vision--language models under incomplete observations. CPP separates task adaptation from proxy reliability through lightweight prompt learning and reliability-aware proxy utilization. Specifically, CPP constructs feature-space proxy representations for missing modalities and conservatively regulates their contribution during multimodal fusion. Experiments on MM-IMDb, Food-101, and Hateful Memes evaluate text-missing, image-missing, and mixed-missing settings under different missing rates. The results demonstrate that CPP achieves robust performance across diverse incomplete multimodal scenarios while requiring only a small number of trainable parameters. Extensive analyses, including ablation, visualization, sensitivity, parameter efficiency, and complete-to-missing evaluations, further verify the effectiveness and generalization capability of the proposed framework.
Source: Research Square
Institutions: Zhejiang Normal University, Sun Yat-sen University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204995493