SCAF-Net: Sparse Cross-Modal Adaptive Fusion Network for RGB-IR UAV Object Detection
Year: 2026
Type: article
Abstract: Visible-infrared (RGB-IR) object detection equips uncrewed aerial vehicles (UAVs) with robust, around-the-clock perception capabilities. However, practical deployments are frequently constrained by weak spatial misalignment between heterogeneous sensors. This issue degrades the object detection accuracy of UAVs, for which a modest offset may cause corresponding features to overlap background regions or neighboring instances. In addition, the substantial appearance difference between RGB and infrared imagery makes direct cross-modal matching unreliable. Inspired by feature decoupling paradigms and cross-modal attention mechanisms, we propose a geometry-guided RGB-IR framework termed the sparse cross-modal adaptive fusion network (SCAF-Net) to improve feature correspondence and interaction under such conditions. Specifically, the modality-decoupled offset estimator estimates modality-shared geometric representations and bidirectional offsets. The geometry-aware sparse cross-attention module uses these offsets to perform saliency-guided sparse cross-modal attention with a coordinate-distance bias, while the context-preserving bidirectional warping mechanism reintegrates the enhanced sparse tokens into dense feature maps within the same reference coordinate domain. The key novelty is the unified offset-guided coordinate flow that consistently guides correspondence estimation, token interaction, and feature reconstruction. Extensive experiments on the DroneVehicle and RGBTDronePerson datasets demonstrate that SCAF-Net achieves competitive performance, exhibiting enhanced robustness in complex aerial-view scenarios.
Authors: Boyu Zhang, Chaoyang Niu, Wanjie Lu, Wei Liu, Zhen Wang
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,800
OpenAlex ID: W7214085331