Joint Global-Local Feature Alignment With Fine-Tuned Pretrained Transformers for Text-Based Person Search
Year: 2025
Type: article
Abstract: Text-based person search (TBPS) aims to retrieve person images from a database using natural language description. Although significant progress has been made, TBPS remains challenging due to the complexities of cross-modal understanding. This work proposes a novel framework named GLAlign that jointly aligns global and local features from both vision and text modalities using large-scale, pretrained, fine-tuned transformers. Specifically, we utilize ViT-B/32 for visual encoding and GPT-2 (English) or PhoBERT (Vietnamese) for textual encoding. To enhance alignment, we perform human part parsing and noun phrase extraction, enabling fine-grained local feature correspondence between body regions and descriptive attributes. The proposed method is evaluated on four benchmark datasets: CUHKPEDES, CUHK-PEDES-VN, 3000VnPerson-Search, and 3000Vn-V2E. In the CUHK-PEDES dataset, our model achieves a Rank-1 accuracy of 80.75%, outperforming state-of-the-art methods such as PLOT (75.28%) and RaSa (76.51%). On the 3000VnPerson-Search dataset, our model reaches a Rank-1 accuracy of 85.72% for Vietnamese descriptions, indicating its robustness across both high-resource and low-resource languages. These results demonstrate the effectiveness of combining global-local alignment with fine-tuned vision language transformers for the TBPS task. The source codes are available at: https://github.com/TextBasedPersonSearch/PersionSearch.
Source: Informatica
Authors: Thi Thanh Thuy Pham, Huong-Giang Doan
Institution: Electric Power University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4413636808