Get started
Work

Does GenAI Perform Better than Students and Novice Software Developers in Agile/Scrum Requirements Engineering Tasks?

Year: 2026
Type: article
Abstract: In Agile/Scrum software development, the product backlog is a key artefact specifying software requirements. However, when starting work in Agile/Scrum settings, many junior software developers struggle with requirements specification, even when detailed requirements elicitation notes are provided. We analysed product backlogs created by 211 student teams to identify common mistake types. On this basis, we elaborated a set of 17 typical mistake types and the corresponding checklist to avoid them. We also analysed the teams’ performance with respect to the identified mistake types in order to determine the most common challenges faced by students. The elaborated checklist and the results of the performance analysis may be useful to educators, students, and junior developers involved in requirements specification. We also analysed the current capabilities of GenAI tools and compared them with the performance of the student teams in similar settings where a product backlog has to be specified based on the detailed notes created within the requirements elicitation process. We conducted experiments with GPT-4.0 Turbo, GPT-6.1 Sol Medium, Gemini 2.0 Flash, Gemini 3.5 Flash-Lite-High, and Val, a version of GPT-4.1 mini refined with university-specific safety requirements. For older models, we present the results from a single run per prompt. For more recent models (GPT-6.1 Sol Medium and Gemini 3.5 Flash-Lite-High), we present the results from three runs per prompt. The experiments allow us to investigate the practical applicability of GenAI tools for drafting the product backlogs and analyse the potential need to adjust settings for the corresponding assignments.
DOI: 10.3390/educsci16111923
Institution: RMIT University
Language: English
Open Access status: gold
Estimated APC paid: $2,172
OpenAlex ID: W7225754150
FWCI:
Cites:
Cited by:
Related to: