Work

A globally applicable deep learning model for Sentinel-2 cloud and shadow detection

Year: 2025

Type: article

Abstract: This study presents and evaluates a globally applicable cloud and shadow masking model for Sentinel-2 top-of-atmosphere (TOA) reflectance using a state-of-the-art transformer-based U-Net model (Swin-Unet) trained with nearly 20 thousand globally distributed 512 × 512 20 m pixel patches to classify each pixel as cloud, cloud shadow, or clear. The training data were compiled from publicly available annotation data, that were refined for obvious annotation errors and supplemented with additional annotations to enhance representation of underrepresented cloud and surface conditions. The trained Swin-Unet model was validated using the KappaSet and CloudSEN12+ testing datasets and compared with the Fmask, Sen2Cor scene classification layer (SCL), and a deep learning model CloudS2Mask. The Swin-Unet achieved the highest overall accuracy (91.32 %) and the highest F1-scores for cloud (0.909) and clear classes (0.935), despite a lower cloud shadow F1-score (0.691) than CloudS2Mask (0.743). The four models were also applied to 11,458 Sentinel-2 images acquired over a calendar year for 78 globally distributed 109 × 109 km tiles and the temporal smoothness index (TSI) of the time series ‘clear’ surface reflectance was derived. The Swin-Unet model yielded the smallest TSI value (most temporally consistent reflectance) for each selected Sentinel-2 band. Visual assessment confirmed the superior performance of the Swin-Unet model. The results highlight the potential of the Swin-Unet model for Sentinel-2 cloud and cloud shadow detection for images acquired anywhere and anytime. The training data and model are publicly available to enable users to apply them efficiently. • Globally applicable Swin-Unet model for Sentinel-2 cloud/shadow detection developed. • Refined and augmented publicly available annotation datasets for training. • Evaluated with 11,458 global images for time series clear reflectance consistency. • Superior accuracy over Fmask, SCL, and CloudS2Mask methods. • Training data, ready-to-use application codes and trained model published.

Source: Science of Remote Sensing

DOI: 10.1016/j.srs.2025.100278

Authors: Dong Luo, Hankui K. Zhang, Hugo De Lemos, Junjie Li, David P. Roy

Institutions: William & Mary, South Dakota State University, Michigan State University

Language: English

Topic: Remote-Sensing Image Classification

Subfield: Media Technology

Field: Engineering

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,180

Funders: South Dakota State University, Endocrine Society of Australia