Work

Parameter-efficient remote sensing image scene classification via mixture of recursive cross-attention and cross-modal semantic initialization

Year: 2026
Type: article
Abstract: This paper introduces Mixture of Recursive Attention for Geographic Encoding (MiRAGE), a unified parameter-efficient approach for Remote Sensing (RS) image scene classification. MiRAGE adapts a Contrastive Language-Image Pre-training (CLIP)-Vision Transformer (ViT) backbone using Low-Rank Adaptation (LoRA). Thus, it reduces fine-tuning cost and addresses overfitting on RS datasets with limited samples. MiRAGE adopts a dual-head architecture, comprising a Global Recursive Head (GRH) and a Spatial Recursive Head (SRH). These heads independently process global contextual cues and local spatial textures. A shared Mixture-of-Recursions (MoR) router governs both heads, dynamically determining an input-image-dependent recursion depth. This makes the computation graph adaptive. A symmetric KL-divergence consistency loss regularizes the predictions of the two heads, encouraging agreement while preserving their complementary representations. We further propose a Cross-Modal Semantic Initialization (CMSI) strategy that uses CLIP text embeddings with class-specific prompts to initialize specific network weights. This enables MiRAGE to transfer semantic priors learned from large-scale multimodal datasets to downstream RS image scene classification. We compare MiRAGE with 92 existing methods on different train-test splits of three datasets. MiRAGE ranks first according to four distinct ranking schemes. Ablation studies indicate that LoRA, MoR, and CMSI contribute to MiRAGE’s performance. Further evaluations show that adaptive routing reduces recursive computation relative to fixed maximum-depth inference. However, measured MiRAGE latency remains higher than baseline CLIP. Experiments also provide evidence of cross-dataset transfer capability for MiRAGE. Further examinations include visualizations of activation maps and embeddings, and a detailed inspection of MoR.
Language: English
Open Access status: gold
Estimated APC paid: $2,340
OpenAlex ID: W7221554657
FWCI:
Cites: 108
Cited by:
Related to: 10