A Multimodal Fusion Framework for Student Engagement Detection Using Facial, Behavioral, and Textual Features
Year: 2026
Type: preprint
Abstract: Student engagement in classrooms has been shown to enhance motivation and improve academic performance; however, maintaining student engagement during instruction remains a significant challenge for educators. This study presents a multimodal framework for student engagement detection by integrating facial, behavioral, and textual cues derived from images, videos, and text data. The proposed framework investigates the contribution of individual modalities and evaluates both feature-level (early) and decision-level (late) fusion strategies for engagement prediction. A key contribution of this work is the analysis of modality-specific influences on engagement detection, providing practical insights for educational applications. Publicly available datasets comprising image, video, and textual data were employed to classify student engagement into three categories: engaged, partially engaged, and disengaged. Experimental results indicate that XGBoost achieved the highest accuracy of 0.867 using handcrafted behav-ioral features, highlighting the predictive significance of observable actions and eye-movement patterns. Furthermore, the late fusion model integrating all three modalities achieved an accuracy of 0.864, outperforming individual modality-based models. The findings demonstrate the effectiveness of multimodal fusion for student engagement detection and highlight its potential for developing intelligent classroom monitoring systems.
Source: Research Square
Institution: COEP Technological University
Language: English
Subfield: Computer Science Applications
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W7214722253