Year: 2025
Type: article
Abstract: In recent years, computer vision and machine learning have achieved remarkable advancements across health care, autonomous systems, and robotics sectors. However, the educational domain, particularly the automated detection of student engagement in online and offline learning environments, remains rich in research opportunities. Detecting student engagement is inherently complex and requires sophisticated interpretive capabilities. This paper introduces a novel approach to automatically detect and classify student engagement by integrating facial images with behavioral and facial features, providing a comprehensive solution to enhance engagement recognition. This study begins by extracting behavioral features and correlating them with pre-defined engagement labels to perform engagement classification using machine learning techniques. In parallel, deep learning models are trained and validated on both image and behavioral features, offering a complementary approach. Additionally, the relationship between facial action units (AUs) and engagement labels is analyzed using three distinct metrics: conditional activation probability, relative activation ratio, and the statistical discriminant coefficient (SDC). The study utilizes publicly available datasets (WACV and DAiSEE) to perform extensive evaluations. Experimental results demonstrate that integrating action units significantly enhances model performance, with XGBoost achieving the highest accuracy (82.9%) among traditional models and EfficientNet reaching the best performance (47.2%) in deep learning experiments. These findings highlight the potential of multimodal approaches in improving real-time engagement detection, offering valuable insights into educational technologies and pedagogy.
Authors: Riju Das, Soumyabrata Dev
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
SDG: Quality education
Open Access status: hybrid
Estimated APC paid: $3,290
Award: ADAPT_Phase 2
OpenAlex ID: W4411769050