Adaptive Hierarchical and Evidential Multimodal Feature Fusion for Aerial Scene and Land-Cover Recognition Under Missing Modalities
Year: 2026
Type: article
Abstract: Recent advances in multimodal data fusion for remote scene classification have significantly improved performance. However, many challenges remain, such as fusing optical and SAR images given their different spatial and temporal characteristics. Prevailing convolutional and transformer-based architectures address these factors only partially: channel and spatial attention are typically modeled independently rather than jointly; multi-branch features are merged through static, content-agnostic concatenation; and a single shared transformation is applied uniformly across all input scales, limiting representational adaptivity. In this paper, we propose a multimodal Adaptive Hierarchical Feature Fusion Network (AHFF-Net), a three-stage, progressive-resolution architecture that unifies four complementary mechanisms to address these limitations jointly. A Spectral-Spatial Gating Mechanism (SSGM) models channel-wise and spatial attention in a single, parameter-efficient formulation to suppress irrelevant channels and background regions simultaneously. Hierarchical Inverted-Residual Dense Blocks (HIRDB) combine inverted-bottleneck expansion, dense inter-layer connectivity, and Fire Module compression to preserve fine-grained detail with a compact parameter footprint. A Progressive Multi-Resolution Inception Cascade (PMRIC) extracts features across parallel receptive fields, and a Semantic Channel Shuffle (SCS) module applies content-adaptive gating with structured interleaving to resolve cross-branch feature isolation. Finally, a Mixture-of-Experts module with Scale-Specific Routing (MoE-SSR) decomposes intermediate features into low-, mid-, and high-frequency bands via a wavelet-inspired decomposition and routes each band to scale-specialized experts using data-driven, Frobenius-norm-based weighting. On three benchmark datasets — MLRSNet, EuroSAT, and NWPU-12 — AHFF-Net achieves classification accuracies of 95.0%, 97.0%, and 94.0%, respectively, outperforming ten pretrained CNN, transformer, and hybrid baselines while requiring roughly 20% fewer parameters and less than one-third the FLOPs of comparable transformer and hybrid architectures, with the lowest per-image inference latency among all evaluated models. Four ablation studies confirm that spectral-spatial gating, hierarchical dense-residual connectivity, and scale-specific expert routing each contribute independently to the accuracy gains while reducing model complexity. For multimodal data, we further propose REMF-Net. This multimodal evidential fusion network encodes hyperspectral, LiDAR, and SAR inputs separately, exchanges information through a center-guided similarity gate and a bottleneck fusion transformer, and combines modality-wise evidential opinions with Dempster's rule, so that it tolerates missing modalities and reports per-sample uncertainty. On Trento, Houston 2013, and Augsburg, REMF-Net reaches overall accuracies of 99.96%, 99.42%, and 99.40%, outperforms five recent multimodal methods, and keeps 99.10% accuracy on Augsburg when only HSI is available. These results establish that adaptive, content-dependent hierarchical fusion is a more effective and computationally efficient alternative to static multi-scale designs, with implications for near-real-time deployment in land-use monitoring and disaster-response mapping.
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,800
OpenAlex ID: W7225324641