Evaluation of the Predictive Accuracy, Interpretability, and Fairness of Machine Learning Methods for STEM Education Data Analysis
Year: 2026
Type: dissertation
Study design: Observational Study
Abstract: Self-report measures of persistence in science, technology, engineering and mathematics (STEM) programs are used to generate guidelines for educational interventions and justify curriculum decisions. However, the validity of these self-report items is often studied using concurrent measures of intent to persist, instead of information about whether students do persist. The present study uses graduation with a STEM degree as an outcome measure to investigate the predictive validity of self-report STEM persistence measures for undergraduate students (N=3072). This analysis evaluates the predictive accuracy, interpretability, and fairness of a logistic regression model, boosting model, and feedforward neural net when investigating item predictive validity in empirical data and a proof-of-concept simulation. Results suggest that the three models have similar predictive accuracy, that responses to the items yield largest differences in predicted probability for the feedforward neural net, and that all three models lack algorithmic fairness for one of five accuracy metrics.
DOI: 10.17615/gz4k-h186
Author: Lewis, Hannah K
Institution: University of North Carolina at Chapel Hill
Language: English
Subfield: Computer Science Applications
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W7162791732
Cites:
Cited by:
Related to: