Work

Impact of AI-assisted programming on a data-driven learning analytics framework for detecting the performance capability gap and cognitive debt in programming education

Year: 2026
Type: article
Abstract: This study examines a performance discrepancy in programming education: students often score higher on AI-assisted coding tasks than on unaided conceptual assessments of similar material. Drawing on the construct of cognitive debt, operationally defined here as the discrepancy between a student’s AI-assisted output and the same student’s ability to demonstrate the underlying concepts without AI assistance, we quantify this gap among 100 students (of 111 enrolled; 50 male, 50 female) in a Bangladeshi university programming course, using an explanatory sequential mixed-methods design. Two performance measures—an AI-assisted practical coding test and an unaided conceptual test—were compared using paired-samples testing, correlation and regression, exploratory k-means cluster analysis (k = 4), and CGPA and gender covariate analysis, supplemented by semi-structured interviews with 10 purposively selected students. Students scored substantially higher on the AI-assisted test than the unaided test. The two measures were only moderately correlated, and cluster analysis identified four exploratory performance profiles, including a Cognitive Debt Zone comprising 34% of students who combined high AI-assisted scores with low conceptual scores. CGPA was positively associated with performance on both tests (r = .55 for each) but negatively associated with the size of the AI-assisted/unaided gap; no significant gender differences emerged, though the study was underpowered to rule out small effects. These findings indicate a measurable performance gap between AI-assisted and unaided assessment that is consistent with a hidden shortfall in conceptual understanding. Because the design is cross-sectional, the two tests differ in format and difficulty, and there is neither a no-AI practical control nor a pre-AI baseline, the study cannot show that skills have accumulated or eroded, cannot attribute the gap to AI use alone, and does not provide a directly validated measure of cognitive debt itself. Building on these results, we propose the AI-Assisted Task–Feynman Technique Cognitive Debt Framework, a preliminary diagnostic and monitoring tool based on these performance-gap indicators, and discuss implications for the quality of AI-integrated programming education (SDG 4).
Language: English
Open Access status: gold
Estimated APC paid: $1,390
OpenAlex ID: W7233601336
FWCI:
Cites:
Cited by:
Related to: