Work

Toward a Quantitative Engagement Monitor for STEM Education

Year: 2024

Type: conference-paper

Abstract: Nearly 50% of college students enrolled into engineering programs in the US drop out after the first year, which covers basic STEM courses as pre-requisites for following courses in various engineering disciplines. A similar trend, but at lower rate, persists with subsequent STEM coursework. This leads to substantial attrition which is costly to students, their families, and society. US institutions have tackled the issue of attrition through advances in the education environment and in the delivery mechanisms of STEM subjects. Despite the progress made, engineering attrition due to performance deficiencies in STEM courses persists across US educational institutions, and is most evident among low income, minorities and first-generation college students. Students' engagement (or lack of) in the classroom is a strong indicator for performance in STEM subjects. Using NSF IUSE Program funding, the investigators proposed to: 1) create an experimental biometric sensor network (BSN) using non-intrusive sensors to capture the emotional and behavioral engagements of the students in the classroom; 2) use computer vision methodologies to extract robust features to quantify the emotional and behavioral engagements, and explore their correlation with cognitive engagements; 3) deploy the BSN in an early engineering STEM class to obtain real-time data for design, test and validate an engagement monitor which would display the results on a teacher's dashboard. A prototype of the BSN based on wireless webcams and wristbands has been created and interfaced with a sever, through which students' facial and physiological information were captured in real-time during the lectures. The facial action coding system (FACS) were identified using a selective facial part model (SPM) developed at the CVIP Lab. Videos have been annotated by an experienced human observer. Features corresponding to the FACS were extracted using a novel convolutional neural network (CNN) to obtain robust descriptors using deep learning, and optimized to correspond to three levels of engagement: highly-engaged, engaged and not engaged. More than 10,000 frames were used in the training of the CNN, and testing was performed using real videos from two STEM classes. Preliminary results are very promising. The BSN enables complete autonomy of the data, and allows for including and excluding students without coercion. The BSN can also be deployed online. A GUI resembling a dashboard has been created to display the engagement levels in real-time. Current efforts are focusing on automating the annotation, and establishing a coherence between the behavioral and emotional engagement for data reduction. Efforts are also directed towards fusing of emotional and behavioral engagement measures and their correspondence to the cognitive engagement of the students.

Source: ASEE Annual Conference & Exposition

DOI: 10.18260/1-2--37916