YOLO-GRAT: A Gated Residual Cross-Modal Fusion Network with Adaptive Triplet Attention for Visible–Infrared Object Detection
Year: 2026
Type: article
Abstract: YOLO-GRAT integrates P3 Cross-Modal Gated Residual Fusion with adaptively weighted Triplet Attention inside backbone C3 bottlenecks of a YOLOv5-based visible–infrared detector. The fusion block combines input-dependent modality mixing, a mean-residual path and pooled-token Transformer refinement; the attention extension learns aggregation weights for three established rotated branches. In the FLIR comparison, each module is added independently to a common dual-stream baseline. Full gives 83.4/44.5% AP50/AP50:95 versus 80.7/42.1% for that baseline. A separate comparison uses four identified fixed-final M3FD seed-0 checkpoints on the same 840 validation pairs. Their official COCO AP is 57.21% for CFT, 57.48% for CGRF only, 57.40% for weighted Triplet only and 56.85% for Full; Full small-object AP is 35.46 versus CFT 35.95%. The FLIR configuration results motivate the combination, while the identified M3FD results reveal metric-dependent trade-offs. The study evaluates complementary within-stream recalibration and cross-modal interaction at the configuration level, with conclusions scoped to the reported datasets, metrics and training records.
Source: Journal of Imaging
Authors: Ting Liu, Xu Yang, Yupeng Jia
Institution: Dalian Maritime University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,172
OpenAlex ID: W7225611656
FWCI:
Cites:
Cited by:
Related to: