Work

GPT-4 as a TA : A Study on Automated Feedback for Introductory Programming Assignments

Year: 2024
Type: dissertation
Abstract: The increasing enrollment rates in computer science programs have posed significant challenges, particularly in providing timely and effective feedback to students in introductory programming courses. This research investigates the potential of OpenAI’s GPT-4, a Large Language Model (LLM), to automate feedback generation for student assignments, thereby alleviating some of the burdens on teaching staff. By evaluating the model’s effectiveness across three metrics—accuracy, correctness, and lucidity—this study aims to determine the feasibility of integrating GPT-4 into educational tools. Student submissions from the course DD1338: Algorithms and Data Structures were selected and artificially injected with common errors. GPT-4 was then prompted to generate feedback based on these submissions using two different prompting methods: one with assignment instructions included and one without. The study found that GPT-4 can identify errors and provide correct advice effectively, with minor differences between the prompting methods. However, the model exhibited a propensity for hallucinations, especially when task instructions were included, thereby impacting the lucidity of the feedback. The results indicate that while GPT-4 shows promise in automating feedback for programming assignments, its tendency to generate hallucinations necessitates human review.
Author: Tohme, Georges
Language: English
Open Access status: green
OpenAlex ID: W7056709574
Cites: 24
Cited by:
Related to: 10