Work

Can Large-Language Models Replace Humans in Agile Effort Estimation? Lessons from a Controlled Experiment

Year: 2024

Type: article

Abstract: Effort estimation is critical in software engineering to assess the resources needed for development tasks and to enable realistic commitments in agile iterations. This study investigates whether generative AI tools, which are transforming various aspects of software development, can improve effort estimation efficiency. A controlled experiment was conducted in which development teams upgraded an existing information system, with the experimental group using the generative-AI-based tool GitLab Duo for estimation and the control group using conventional methods (e.g., planning poker or analogy-based planning). Results show that while generative-AI-based estimation tools achieved only 16% accuracy—currently insufficient for industry standards—they offered valuable support for task breakdown and iteration planning. Participants noted that a combination of conventional methods and AI-based tools could offer enhanced accuracy and efficiency in future planning.

Source: Applied Sciences

DOI: 10.3390/app142412006

Authors: Luka Pavlič, Vasilka Saklamaeva, Tina Beranič

Institution: University of Maribor

Language: English

Topic: Software Engineering Techniques and Practices

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,728

Funder: Javna Agencija za Raziskovalno Dejavnost RS