A Reliability Audit of AI-Assisted Movement Assessment for Physical Education: A Reproducible Public-Data Stress Test of Movement Classifiers
Year: 2026
Type: article
Abstract: AI-assisted movement assessment is being considered for physical education, but such systems must remain dependable when observations degrade. This study provides a reliability audit of movement classification before any classroom adoption, treating trustworthy behavior under imperfect sensing as a precondition for AI-supported instruction. Six fixed predictors, including a predefined random forest, were evaluated on 2000 University of Idaho Physical Rehabilitation Movement Data (UI-PRMD) repetitions from ten healthy participants, with five participant-separated folds and controlled perturbations that preserve the source labels. The audit separates three failure layers that accuracy alone conflates: ranking quality, fixed-threshold decisions, and probability calibration. The forest reached 83.6% clean balanced accuracy; temperature scaling improved calibration; and a quality-aware deferral rule withheld unreliable predictions, a mechanism we interpret as a teacher-review trigger rather than an automated grading decision. Under only 2-degree independent angular noise, fixed-threshold decisions collapsed (52.8%) while ranking information largely survived (AUC 0.855), and severity-matched cutpoint adaptation recovered most decisions (77.0%). Classification was stable under 30% missing rotations and 10-Hz downsampling. Sensitivity analyses over the deferral-score coefficients, temperature stability across folds, and leave-one-subject-out influence confirmed these conclusions are not artifacts of particular parameter choices. The audit is fully reproducible and dataset-specific; educational effectiveness, classroom transfer, and physical-sensor robustness remain untested and are prerequisites for instructional deployment.
Source: Applied Sciences
DOI: 10.3390/app162110908
Institution: National University of Malaysia
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W7225192019