GLFNet: An RGB-T Crowd Counting Network Based on Global–Local Multimodal Feature Fusion
Year: 2025
Type: article
Abstract: Multi-modal fusion is a prevalent method in crowd counting tasks, capable of garnering comprehensive crowd information via the amalgamation of features from varying modalities. This approach is particularly beneficial in environments characterized by uneven lighting and low visibility of crowds. In such cases, integrating Red Green Blue (RGB) and Thermal features can augment the representation ability of crowd features, thereby boosting the accuracy of crowd counting. However, the full integration of multi-modal information remains a challenge that researchers continue to investigate. This paper introduces an innovative RGB-Thermal (RGB-T) crowd counting network named Global Local Feature Fusion Network (GLFNet), which differs considerably from existing RGBT methodologies. GLFNet prioritizes both global and local feature relationships between modalities. Specifically, in terms of global multi-modal feature fusion, a learnable block count token (BCT) is constructed to guide the interaction of two types of modal information; in terms of local multi-modal fusion, we have designed a novel Composite Spatial Attention Mechanism (CSAM). CSAM emphasizes the spatial distribution of crowds across both modalities and provides detailed information supplementing global multi-modal features. Further, we devised and incorporated a global-local consistency loss (Lc) during the training process, ensuring uniformity between global and local crowds estimates. Experimental results on the RGBT-CC and DroneRGBT datasets indicate our method achieves optimal performance to date.
Authors: Yingxiang Hu, Yanbo Liu, Guo Cao, Jin Wang
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G4993641449, G7776482482
OpenAlex ID: W4409427764