VTS-PANDA: fine-grained wild giant panda behavior recognition via ecological semantic knowledge from infrared videos
Year: 2026
Type: article
Abstract: Recognizing fine-grained giant panda behaviors from long-term infrared camera-trap videos remains challenging due to subtle motion variations, complex natural environments, and the inability of vision-only models to exploit high-level semantic information. To address these challenges, we propose VTS-PANDA, a unified multimodal framework that jointly models temporal visual dynamics and ecology-aware semantic priors. The framework introduces three complementary components: a depthwise temporal convolution module that enhances local inter-frame motion cues at the input stage, an MLLM-guided semantic prior module that generates structured ecological descriptions through constrained prompting and integrates them with visual features via a gated fusion mechanism, and a CLIP-based similarity enhancement module that explicitly aligns visual representations with behavioral text prompts to improve discriminability. Extensive experiments on wild giant panda infrared video datasets demonstrate that the proposed framework effectively captures temporal motion patterns, spatial structures, and high-level ecological semantics, achieving robust and interpretable behavior recognition in challenging real-world environments. This work provides a semantic-guided framework for automated wildlife behavior understanding and offers potential support for intelligent long-term ecological monitoring and precision conservation.
Authors: Nuo Xu, Yanqi Dong, Jiali Zi, Yi Xia, Feixiang Chen
Institution: Beijing Forestry University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,990
Award: G4538521706
OpenAlex ID: W7203869467