Design, Implementation, and Empirical Evaluation of a Human–AI Collaborative Adaptive Assessment Model in Higher Education
Year: 2026
Type: article
Abstract: AI-supported assessment increasingly enables adaptive tasks, automated analysis, and personalized feedback, yet their integration into coherent, human-supervised formative assessment remains insufficiently established. This study developed and evaluated the Human–AI Collaborative Adaptive Assessment Model (HA-CAAM) using Design and Development Research structured through the ADDIE framework. Implemented in a higher-education Media Education course, the model employed performance-responsive task adaptation using predefined thresholds, rubric-guided AI-assisted scoring, formative feedback, and instructor verification. Five experts validated the model, 148 students provided quantitative and qualitative implementation feedback, and 50 student responses were independently assessed by two human raters and AI. Expert validation provided initial evidence of content validity (mean Aiken's V = 0.818). Students reported generally favorable experiences, particularly regarding clarity and feedback usefulness, although Learning Support and Overall User Experience received comparatively lower ratings. AI-generated scores demonstrated strong agreement with human-reference scores (ICC = 0.898), despite individual discrepancies. These findings provide initial evidence of content validity, positive implementation experiences, and scoring agreement, rather than demonstrated improvements in learning outcomes or equivalence between AI and human assessment decisions. HA-CAAM contributes an empirically refined, seven-stage assessment framework that coordinates performance-responsive adaptation, AI-assisted analysis, formative feedback, and instructor-led quality assurance within a human-supervised assessment process.
Language: English
Subfield: Education
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
OpenAlex ID: W7225708534
FWCI:
Cites:
Cited by:
Related to: