Multimodal Satellite-Field Data Integration for Scalable Crop-Type Mapping With Limited Labels
Year: 2025
Type: article
Abstract: Crop type mapping is crucial for improving crop yield predictions, informing agricultural policy decisions, and addressing the impacts of climate change. Despite achieving state-of-the-art performance, current transformer-based architectures that use Satellite Image Time Series (SITS) require costly labelled data and struggle to generalize to new regions. To address this limitation, this paper presents a method that leverages multimodal information from field images and satellite data to address the scarcity of reference data in crop type mapping. The proposed approach consists of three main steps: (1) multimodal data preparation, (2) multimodalpseudo-labelling, and (3) SITS-based crop type mapping. The first step processes both data sources for facilitating their integration, while the novelty lies in the second step, where additionalpseudo-labels are generated by classifying crop parcels with both SITS and field image availability, leveraging multimodal information to produce reliable label predictions. Finally, the third step produces a crop type map for the entire study area using the SITS and the enriched training set. Experimental results at the European level, using the 2018 Land use and land cover survey (LUCAS) field images paired with Sentinel-2 SITS, demonstrate the effectiveness of integrating satellite top-view data with ground-level field images. Thepseudolabelled data achieves performance comparable to models trained on the complete labelled dataset, while substantially reducing the need for extensive labelled data collection.
Authors: Adel Abbas, Claudia Paris, Andrew Nelson
Language: English
Subfield: Ecology
Field: Environmental Science
Domain: Physical Sciences
SDG: Zero hunger
Open Access status: green
OpenAlex ID: W4411408473