A Meta-Analysis of LLM Effects on Students across Qualification, Socialisation, and Subjectification
Year: 2025
Type: review
Abstract: Large language models (LLMs) are increasingly positioned as solutions for education, yet evaluations often reduce their impact to narrow performance metrics. This paper reframes the question by asking "what kind of impact should LLMs have in education?" Drawing on Biesta's tripartite account of good education: qualification, socialisation, and subjectification, we present a meta-analysis of 133 experimental and quasi-experimental studies (k = 188). Overall, the impact of LLMs on student learning is positive but uneven. Strong effects emerge in qualification, particularly when LLMs function as tutors in sustained interventions. Socialisation outcomes appear more variable, concentrated in sustained, reflective interventions. Subjectification, linked to autonomy and learner development, remains fragile, with improvements confined to small-scale, long-term studies. This purpose-level view highlights design as the decisive factor: without scaffolds for participation and agency, LLMs privilege what is easiest to measure while neglecting broader aims of education. For HCI and education, the issue is not just whether LLMs work, but what futures they enable or foreclose.
Source: arXiv (Cornell University)
Authors: Huang, Jiayu, Wang, Ruoxin Ritter, Liu, Jen-Hao, Xia, Boming, Huang, Yue, Sun, Ruoxi, Xue, Jason Minhui, Zou, Jinan
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W7084447516
FWCI:
Cites:
Cited by:
Related to: