Work

Evaluating LLMs for Automated Scoring in Formative Assessments

Year: 2025

Type: article

Abstract: The increasing complexity and scale of modern education have revealed the shortcomings of traditional grading methods in providing consistent and scalable assessments. Advancements in artificial intelligence have positioned Large Language Models (LLMs) as robust solutions for automating grading tasks. This study systematically compared the grading performance of an open-source LLM (LLaMA 3.2) and a premium LLM (OpenAI GPT-4o) against human evaluators across diverse question types in the context of a computer programming subject. Using detailed rubrics, the study assessed the alignment between LLM-generated and human-assigned grades. Results revealed that while both LLMs align closely with human grading, equivalence testing demonstrated that the premium LLM achieves statistically and practically similar grading patterns, particularly for code-based questions, suggesting its potential as a reliable tool for educational assessments. These findings underscore the ability of LLMs to enhance grading consistency, reduce educator workload, and address scalability challenges in programming-focused assessments.

Source: Applied Sciences

DOI: 10.3390/app15052787

Authors: Pedro C. Mendonça, Filipe Quintal, Fábio Mendonça

Institutions: Instituto de Tecnologias Interativas, Universidade da Madeira

Language: English

Topic: Artificial Intelligence in Education

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: gold

Estimated APC paid: $2,896