Reliability-Aware Cross-Modal Fusion of Sentinel-1 SAR and Sentinel-2 Optical Imagery for Robust Patch-Level Land Cover Scene Classification
Year: 2026
Type: article
Abstract: Fusing Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 optical imagery is challenging when one modality is degraded. This study proposes a reliability-aware cross-modal fusion framework for patch-level land cover scene classification, combining modality-specific encoders, per-modality classification heads, and a noise-aware reliability gate that estimates adaptive fusion weights from feature representations, prediction confidence, and optical quality indicators. Sentinel-2 inputs were synthetically degraded under five controlled scenarios-Gaussian noise, heavy Gaussian noise, channel drop, cloud-like masking, and mixed degradation and six approaches were compared: SAR-only, optical-only, early fusion, feature-level fusion, logit-level fusion, and the proposed method. Experiments on a four-class SEN12MS-derived dataset (16,000 paired patches) were run under deterministic settings and reported over five random seeds. The proposed model achieved the highest or statistically comparable mean accuracy under clean, channel-drop, and cloud-mask conditions and a clear advantage under Gaussian noise (0.922 ± 0.051, versus 0.651 ± 0.201 for logit-level fusion and 0.293 ± 0.096 for optical-only) and remained the highest or statistically comparable method on clean inputs. Heavy noise remained the most challenging and seed-sensitive condition (0.604 ± 0.236 across seeds), although the representative run retained 86.3% accuracy, and a scene-held-out evaluation showed that the noise-robustness advantage persists under spatially independent testing. These results position reliability-aware fusion as a robustness-oriented strategy supported by multi-seed statistics and explicit failure-mode analysis.
Source: Remote Sensing
DOI: 10.3390/rs18152484
Authors: Srinivas Vulli, Ming Liu, Zhenyu Huang
Institution: Central Michigan University
Language: English
Subfield: Media Technology
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,258
Funder: Central Michigan University
OpenAlex ID: W7171805494