Deep-Learning-Based Human Activity Recognition: Eye-Tracking and Video Data for Mental Fatigue Assessment
Year: 2025
Type: article
Abstract: This study addresses mental fatigue as a critical state arising from prolonged human activity and positions its detection as a valuable task within the broader scope of human activity recognition using deep learning. This work compares two models for mental fatigue detection: a model that uses eye-tracking data for fatigue predictions and a vision-based model that relies on vital signs and human activity indicators from facial video using deep learning and computer vision techniques. The eye-tracking model (based on TabNet architecture) achieved 82% accuracy, while the vision-based model (features were estimated using deep learning and computer vision) based on Random Forest architecture reached 78% accuracy. A correlation analysis revealed strong alignment between both models’ predictions, with 21 out of 27 sessions showing significant positive correlations on the collected dataset. Further comparison with an earlier-developed vision-based model trained on another dataset supported the generalizability of the vision-based model using physiological indicators extracted from a facial video for fatigue estimation. These findings highlight the potential of the vision-based model as a practical alternative to sensor and special-devices-based systems, especially in settings where non-intrusiveness and scalability are critical.
Source: Electronics
Institution: Russian Academy of Sciences
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W4414517274