Get started
Work

ARA-RSIC: Adaptive Retrieval-Augmented Large Vision-Language Model for Remote Sensing Image Captioning

Year: 2026
Type: article
Abstract: Recent advances in large vision-language models (LVLMs) have significantly improved multimodal understanding and generation. However, adapting these foundation models to specialized domains remains challenging due to domain-specific knowledge gaps, limited reasoning over fine-grained concepts, and insufficient access to external knowledge. Remote sensing image captioning (RSIC) represents a challenging scenario where general-purpose LVLMs often struggle to capture specialized semantics and complex spatial relationships. Retrieval-Augmented Generation (RAG) enables LVLMs to use external examples, but simple similarity-retrieval prompting does not explicitly coordinate context diversity, learned evidence weighting, and selective caption revision. In this work, we propose ARA-RSIC, an Adaptive Retrieval-Augmented Generation framework that enhances LVLM adaptation and multimodal reasoning for remote sensing scenarios. The framework’s contribution lies in coordinating three task-specific mechanisms: image-query-based cross-modal candidate reranking and adaptive context selection in MMCR; query- and retrieval-conditioned gated cross-attention in the Qwen2VL-7B generator; and confidence-triggered context revision with a constrained caption acceptance rule. RemoteCLIP embeddings, MMR, and LoRA are adopted as established building blocks rather than claimed as new algorithms. This integrated design enables selective retrieval, explicit learned weighting of retrieved evidence, and targeted revision beyond fixed-example prompting. Experiments on UCM-Captions, Sydney-Captions, and RSICD show strong BLEU performance and improvements over controlled retrieval-and-generation ablations. On RSICD, the highest BLEU scores coexist with lower ROUGE_L and CIDEr scores than those of selected competing methods, revealing metric-dependent trade-offs in remote sensing captioning.
Source: Entropy
DOI: 10.3390/e28111242
Language: English
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7225821175
FWCI:
Cites:
Cited by:
Related to: