Work

JorGPT: Instructor-Aided Grading of Programming Assignments with Large Language Models (LLMs)

Year: 2025

Type: article

Abstract: This paper explores the application of large language models (LLMs) to automate the evaluation of programming assignments in an undergraduate “Introduction to Programming” course. This study addresses the challenges of manual grading, including time constraints and potential inconsistencies, by proposing a system that integrates several LLMs to streamline the assessment process. The system utilizes a graphic interface to process student submissions, allowing instructors to select an LLM and customize the grading rubric. A comparative analysis, using LLMs from OpenAI, Google, DeepSeek and ALIBABA to evaluate student code submissions, revealed a strong correlation between LLM-generated grades and those assigned by human instructors. Specifically, the reduced model using statistically significant variables demonstrates a high explanatory power, with an adjusted R2 of 0.9156 and a Mean Absolute Error of 0.4579, indicating that LLMs can effectively replicate human grading. The findings suggest that LLMs can automate grading when paired with human oversight, drastically reducing the instructor workload, transforming a task estimated to take more than 300 h of manual work into less than 15 min of automated processing and improving the efficiency and consistency of assessment in computer science education.

Source: Future Internet

DOI: 10.3390/fi17060265

Authors: Jorge Cisneros-González, Natalia Gordo-Herrera, Iván Barcia-Santos, Javier Sánchez-Soriano

Institution: Universidad Francisco de Vitoria

Language: English

Topic: Teaching and Learning Programming

Subfield: Computer Science Applications

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: gold

Estimated APC paid: $1,930