Can Scene-Level NWP Cloud Cover Replace Expert Pixel Labels? A Large-Scale Assessment with ERA5, GFS and CloudSEN12+
Year: 2026
Type: article
Abstract: Deep learning cloud detectors require large volumes of pixel-level labels that are expensive to produce. Gridded cloud cover from ERA5 reanalysis and GFS forecasts could serve as freely available weak supervision. We assess this hypothesis through a large-scale comparison of NWP total cloud cover with expert-annotated cloud masks from CloudSEN12+ (Sentinel-2, 10m) over 48,000 globally distributed samples (2017–2022). ERA5 shows moderate agreement (r=0.42, MAE ∼30%), with performance strongly dependent on cloud type. We then train U-Net cloud segmentation models that differ only in the source of their labels, on a region-disjoint partition of the dataset, and evaluate them on 975 held-out expert-annotated patches. Expert labels yield a binary cloud IoU of 0.78 and a scene-level accuracy of 0.92, against 0.47 and 0.71 for ERA5-derived weak labels. Increasing the weak supervision from 200 to 20,000 patches produces no systematic improvement over the tested range, which suggests that, for this label formulation, the limitation lies in what each label expresses rather than in how many are available; 849 expert-annotated patches outperform 20,000 weakly labelled ones. Pretraining on ERA5 labels and fine-tuning on expert masks is worse than training from scratch at every annotation budget that we tested. Training on ERA5 labels does improve on applying the same labels directly as a mask, from 0.38 to 0.47 in binary cloud IoU, and the resulting model remains usable for coarse scene-level screening where no pixel-level annotation exists. Results provide practical guidelines for leveraging NWP products in deep learning training pipelines where expert annotations are scarce.
Source: Applied Sciences
DOI: 10.3390/app162211444
Authors: Anna Anzalone, Antonio Pagliaro
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
SDG: Climate action
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W7235915779
