Work

A Light-Weighted Fusion Vision Mamba for Multimodal Remote Sensing Data Classification

Year: 2025

Type: article

Abstract: Recent studies have shown that vision Mamba (VMamba) excels in long-sequence modeling capabilities, offering efficient visual representation learning. However, the existing VMamba-based methods primarily focus on single modality and are not readily adaptable for multimodal data processing. In this study, we aim to leverage the power of VMamba by investigating a light-weighted fusion VMamba for multimodal remote sensing data classification. First, to integrate information from various modalities, we propose a spatial and channel fusion VMamba for multimodal remote sensing classification. For spatial fusion, a two-branch state-space model is constructed based on VMamba, where the parameters of each branch interact to merge the spatial information from different modalities. Regarding channel fusion, a channel fusion VMamba is introduced for multimodal remote sensing data classification, which employs a specific eigenvalue computation in the frequency domain for more effective feature fusion based on the fast Fourier transformation. Second, to minimize the computational cost of the fusion VMamba in multimodal remote sensing data classification, we explore a light-weighted fusion VMamba. Specifically, information from different modalities is reconstructed by adopting a skip sampling scanning scheme within VMamba, which replaces the standard scanning scheme and reduces the number of parameters in VMamba. Extensive experiments on three public multimodal remote sensing datasets have demonstrated that our proposed light-weighted fusion VMamba surpasses state-of-the-art methods in terms of classification accuracy and computational cost. For instance, the proposed light-weighted fusion VMamba achieves a 20% reduction in FLOPs compared with the standard VMamba on the Houston dataset for multimodal remote sensing classification.

Source: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

DOI: 10.1109/jstars.2025.3598755

Authors: Xin He, Xiao Han, Yushi Chen, Lingbo Huang

Institution: Harbin Institute of Technology

Language: English

Topic: Remote-Sensing Image Classification

Subfield: Media Technology

Field: Engineering

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,800

Funders: National Natural Science Foundation of China, Natural Science Foundation of Heilongjiang Province