Work

Multi-grained image-text retrieval in farmland remote sensing via multimodal large language models

Year: 2026

Type: article

Abstract: Low-altitude farmland remote sensing (FRS) has become an important tool for precision agricultural monitoring. However, existing FRS methods are mainly designed for segmentation with predefined categories, making them insufficient for semantic understanding and unable to support flexible natural-language queries. To address this limitation, we introduce an artificial intelligence-based task, FRS cross-modal retrieval, which aims to retrieve target farmland regions from large-scale remote sensing images using textual descriptions. To support this task, we construct the first Farmland Remote Sensing Cross-Modal Retrieval (FRS-CMR) dataset, containing 67,806 image-text pairs generated through multimodal large language models and expert refinement. Based on FRS-CMR, we propose a Farmland-oriented Multi-Grained Retrieval (FarmMGR) framework for image-text alignment. FarmMGR jointly exploits global scene semantics and local region-level cues to capture both large-scale landscape patterns and fine-grained farmland characteristics. To further improve semantic alignment, we design Prototype-Based Semantic Alignment (PBSA) to alleviate the soft-positive sample problem, Global Semantics Hierarchical Feature Integration (GSHFI) to adaptively fuse global and local representations, and Instance-Level Semantic Relation Modeling (ISRM) to capture relational dependencies among farmland instances. In addition, Memory-Enhanced Cross-Modal Contrastive Learning (MECCL) enlarges the negative sample pool and improves representation discrimination. Extensive experiments on FRS-CMR show that FarmMGR achieves a mean recall of 28.57%, outperforming existing image-text retrieval methods and demonstrating its effectiveness for language-driven farmland remote sensing retrieval.

Source: Engineering Applications of Artificial Intelligence

DOI: 10.1016/j.engappai.2026.116113