Leveraging Large Language Models for Assessments Grading and Personalised Feedback Generation: A Case of Software Engineering
Year: 2026
Type: preprint
Abstract: The emergence of Artificial Intelligence and Large Language Models (LLMs) is transforming the education sector. Researchers have recently started exploring LLMs for evaluating educational assessments and generating personalised feedback; however, their evaluation and effectiveness in evaluating complex Software Engineering artefacts remain underexplored. In the proposed approach, we used LLMs, including ChatGPT-4 and Gemini 2.5, to identify rubric-based grading of students’ assessments and to generate personalised feedback using integrated zero-shot and chain-of-thought prompting. For this purpose, 54 students’ assessments from the ‘Software Design and Architecture’ course were evaluated, including UML-based artefacts such as use case diagrams, class diagrams, and three-tier architecture. Each assignment was evaluated using an instructor-defined rubric, and two different LLMs generated scores. The LLMs’ generated scores were compared with the instructor-defined grades using Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Pearson correlation, Spearman correlation, error tolerance analysis, and paired-samples (t)-tests. Student perceptions of AI-generated feedback were also examined through a questionnaire-based survey. The results show that both LLMs generated structured rubric-based evaluations and meaningful feedback. Gemini 2.5 achieved a slightly lower MAE than GPT-4, while GPT did not show a significant mean-level difference from instructor scores. GPT-4 was also highly preferred by the students for feedback quality, personalization, and similarity to the instructor’s feedback. However, both models demonstrated weak correlation and score compression with instructor grading. These findings suggest that LLMs can provide effective feedback and preliminary grading, but should not replace human instructors in high-stakes assessments. A hybrid assessment model where LLMs assist instructors, while the final authority remains with the instructor, is therefore recommended. The proposed approach is supported by a web-based tool.
Source: Research Square
Language: English
Subfield: Education
Field: Social Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W7204195449