Year: 2024
Type: dissertation
Abstract: Worked examples (WEs) are step-by-step solutions that guide learners through a solution process. They have been proven effective at teaching problem-solving and are widely used in education. However, the production of WEs is time-consuming. Hence, instructors typically create few WEs for students to learn from. To address this production problem, Large Language Models (LLMs) have been proposed as a solution. A study from early 2024 claims to be the first to investigate the application of LLMs in generating WEs in computer science education. They found that the LLM they used could generate overall high-quality WEs of CS1 programming questions. Our study used a similar method to generate and evaluate WEs of CS2 programming questions. An LLM was used with three selected CS2-level questions and a designed prompt to generate fifteen WEs. A subjective evaluation by the authors revealed that most WEs were of high quality, often only needing minor modifications. Importantly, all generated WEs had code that compiled without compile-time errors and most solutions gave the correct output. All WEs also contained a step-by-step solution with code and written explanations that displayed high coherence and readability. Common weaknesses were a lack of commented code and that some steps were considered meaningless as separate steps. The LLM also had difficulties in including sufficient explanations related to time complexity. In conclusion, although conducted with limited scope, our results show the value of using LLMs to generate WEs. Proposed future work includes investigating how the results could be improved with iterative feedback to the LLM.
Authors: Lindvall, Vincent, Marcus, Sofia
Language: English
Subfield: Computer Science Applications
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: green
Funder: Kungliga Tekniska Högskolan
OpenAlex ID: W7062816744