Work

A Role-Aware Explainable AI Dashboard for At-Risk Student Classification: Design and Technical Evaluation on the Open University Learning Analytics Dataset

Year: 2026
Type: preprint
Abstract: Predictive models built on learning management system (LMS) data can flag students at risk of failing or withdrawing, but most systems present risk scores without their reasoning and give every staff role the same view. This study designed, implemented and technically evaluated a role-aware explainable artificial intelligence (XAI) dashboard that delivers explanations from one prediction system to instructors, academic advisors and administrators at different levels of aggregation. Using design science research and the CommonKADS knowledge engineering methodology, three ensemble classifiers were evaluated on the Open University Learning Analytics Dataset (32,593 student-module-presentation observations) with four-fold cross-validation grouped by course presentation; SHAP and LIME formed the explanation layer, and the prototype was integrated with Moodle and Canvas. The XGBoost prototype, selected on recall, achieved out-of-fold recall of 0.909, precision of 0.866, F1-score of 0.887 and AUC of 0.962, with accuracy ranging from 0.846 to 0.916 across held-out presentations. Both explanation methods ranked assessments submitted first and shared seven of their top ten features, but the mean per-student top-5 overlap was only 0.635, supporting the display of both methods side by side. Because the features span the full presentation, the results describe retrospective classification; practitioner trust, calibration and fairness remain to be evaluated.
Language: English
Open Access status: green
OpenAlex ID: W7234835232
Cites:
Cited by:
Related to: 10