Evaluating ChatGPT's Reliability in Second Language Acquisition (SLA): Insights on Language Skills and Technology's Role
Year: 2025
Type: article
Abstract: Artificial Intelligence (AI) has become a transformative tool across various fields, including education, offering innovative opportunities for second language (L2) learning and teaching. The potential of AI-driven tools like ChatGPT in Second Language Acquisition (SLA) remains an area of emerging interest, with implications for enhancing pedagogical approaches and providing scalable, interactive support for learners and educators. This study evaluates the reliability and quality of ChatGPT-4.0’s responses to 48 key questions related to SLA, encompassing themes such as technology's role and language skills (reading, writing, listening, and speaking). The questions were rated by five expert linguists using a Likert scale to assess accuracy, and relevance. Statistical analyses, including the Kruskal-Wallis H test (p < 0.05) and post-hoc pairwise comparisons, revealed significant differences among evaluators (p < 0.001), with most responses rated as good (26.25%) or very good (55.41%). Despite these positive outcomes, Fleiss’s Kappa coefficient (0.006) highlighted low inter-rater agreement, signaling a need for standardized evaluation frameworks. While ChatGPT shows promise in addressing diverse SLA topics, its limitations—such as occasional inaccuracies—underscore the necessity for ongoing research to optimize its utility in educational contexts. This study contributes to SLA pedagogy by demonstrating the potential and boundaries of integrating AI into language learning, advocating for cautious and evidence-based adoption.
Authors: Albatool Ahmad Alhazmi, Muneera Muftah
Institution: Najran University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: diamond
OpenAlex ID: W4408308351