Work

Grading Open‐Ended Questions Using LLMs and RAG

Year: 2025

Type: article

Abstract: ABSTRACT Evaluating open‐ended questions is a common and time‐consuming task in education. With the continuous advances in natural language processing (NLP), large language models (LLMs) trained on massive datasets can assist in this process. This study evaluates the use of LLMs, complemented by retrieval‐augmented generation (RAG), for the numerical grading of open‐ended answers of approximately 250 words. We focus on two Spanish‐language technical courses and assess general‐purpose LLMs. Our results show that RAG improves grading accuracy, achieving reductions in mean absolute error (MAE) of up to 19.47% compared to using LLMs alone, with the best configuration reaching a MAE of 1.19. We also note that LLMs tend to assign high grades, reflecting the dataset's imbalance toward higher scores. This work demonstrates the potential of combining RAG with general‐purpose LLMs to evaluate specialised Spanish language content, avoiding the cost and bias of model fine‐tuning.

Source: Expert Systems

DOI: 10.1111/exsy.70174

Authors: Jacobo Farray Rodríguez, Antonio Jesús Fernández‐García, Elena Verdú

Institutions: Universidad Internacional de La Rioja, University of Almería

Language: English

Topic: Artificial Intelligence in Education

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: gold

Estimated APC paid: $3,600