Work

Proaktiivisen ohjelmointiassistentin kehittäminen, joka hyödyntää suuria kielimalleja yksilöllisen ja reaaliaikaisen palautteen tarjoamiseksi

Year: 2025
Type: dissertation
Abstract: Formative feedback is essential for effective learning and undeniably improves learning outcomes when timely and personalized. While face-to-face feedback has proven effective in traditional classrooms, Massive Open Online Courses (MOOCs) lack this privilege. This reveals a gap: the absence of tools that provide personalized real-time feedback while coding. This thesis addresses this gap by developing and demonstrating a proof-of-concept artifact using Design Science Research Methodology (DSRM). The proposed artifact operationalizes these typical feedback scenarios when experts provide during the programming process and uses Large Language Models (LLMs) for personalized feedback generation. The developed artifact uses an event-driven architecture that leverages NATS queues, WebSockets, lightweight Deno services, and parallel processing on the client side to meet the scalability demands of a MOOC environment. In a small-scale evaluation of the artifact, the static analysis feedback was perceived as useful (3.7/5), timely (3.6/5), and clear (3.4/5). In contrast, GPT-4o generated hints received lower ratings (usefulness 2.8/5, timeliness 2.0/5, clarity 2.8/5) due to 20–30 seconds latency and unsolicited feedback. Based on these findings, participants generally preferred pulled or hybrid feedback mechanisms rather than proactive pushed feedback. These findings indicate that compiler-style real-time feedback of the artifact provided by static analysis, in comparison, better addresses students' immediate needs than LLM-generated hints, which add value primarily when delivered promptly or under learner control. Furthermore, the proof-of-concept demonstrates the technical feasibility of operating at scale through design decisions based on microservices and an event-driven architecture. However, further improvements are necessary to reduce feedback latency and improve strategies to trigger feedback generation.
Language: English
Open Access status: green
OpenAlex ID: W7052092492
Cites: 80
Cited by:
Related to: 10