Dual-branch multimodal scale-interaction transformer for semi-supervised HSI-SAR classification
Year: 2026
Type: article
Abstract: Deep learning has significantly advanced hyperspectral image (HSI) and synthetic aperture radar (SAR) joint classification, achieving promising land-cover mapping performance. However, most existing methods rely heavily on dense pixel-level annotations, which are costly and time-consuming to acquire in practical multimodal remote sensing applications. In this article, we propose a novel semi-supervised framework, called Dual-Branch Multimodal Scale-Interaction Transformer (DMSIT), for HSI-SAR joint classification with limited labeled samples. Specifically, the proposed DMSIT first employs dual-stream convolutional encoders coupled with a Cross-Modal Feature Separation (CMFS) module. The CMFS module explicitly disentangles modality-shared classification features from sample-specific private representations, thereby effectively reducing cross-modal feature interference. Subsequently, the separated shared features are transformed into token sequences and processed by a Multimodal Scale-Interaction Transformer (MSIT). By executing scale-aware attention, MSIT captures global cross-modal dependencies and adaptively integrates complementary multimodal features, compensating for the limited receptive fields of local convolutions. Furthermore, to exploit unlabeled data, we introduce a Graph-Guided Pseudo-Labeling (GGPL) strategy. By leveraging graph topological proximity rather than isolated prediction confidences, GGPL selects reliable pseudo-labels, effectively mitigating class imbalance and suppressing error accumulation. Extensive experiments on three datasets demonstrate the effectiveness and superiority of the proposed DMSIT in comparison with several state-of-the-art methods.
Institution: Xi'an Technological University
Language: English
Subfield: Media Technology
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,990
OpenAlex ID: W7223127489