UAPE-CLIP: Uncertainty-Aware prompt learning with vision–language alignment for student engagement recognition
Year: 2026
Type: article
Abstract: Student engagement recognition underpins the understanding of learning behaviors, early warning of learning risks, and the design of instructional interventions. Most existing approaches primarily model visual spatiotemporal cues and make limited use of semantic priors encoded in label text, which can lead to brittle performance when cues are weak, states co-occur, or the deployment domain shifts. To address these challenges, we propose UAPE-CLIP (Uncertainty-Aware Prompt Learning for Student Engagement Prediction via Vision-Language Alignment). UAPE-CLIP leverages cross-modal semantic supervision during training while adopting a decoupled video-only inference pathway to avoid reliance on label text at deployment. We introduce an uncertainty-aware task-adaptive prompt learning mechanism that scales soft prompts according to prototype uncertainty, stabilizing state semantics and improving video–text alignment. To mitigate pseudo-negative interference caused by repeated semantic prototypes in multi-label settings, we employ a multi-positive contrastive objective. Multi-grained cross-modal matching is further combined with temporal modeling to capture both global behavioral patterns and key-frame evidence. Experiments on DAiSEE and AFEW show that UAPE-CLIP achieves 70.63% accuracy on DAiSEE (a +3.22-point gain over the strongest baseline) and 64.75% on AFEW (a +2.04-point gain over the best video-only contrastive method). Overall, UAPE-CLIP offers a practical balance among accuracy, computational cost, and deployment consistency for engagement analytics.
Authors: Yong Lu, Liangtong Jiang
Institutions: Hubei Engineering University, Fujian Normal University
Language: English
Subfield: Computer Science Applications
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: gold
Estimated APC paid: $1,990
OpenAlex ID: W7204220563