Year: 2025
Type: conference-paper
Abstract: Large Language Models (LLMs), combined with advanced prompting techniques, have been used in requirements engineering, particularly in the automated generation of user stories. These stories are essential for agile software projects, but manual creation can be time-consuming and prone to inconsistencies, which has driven interest in automated approaches. Questions about the effectiveness and practical acceptance of these approaches, especially regarding quality and software professionals’ perceptions, still remain. Furthermore, little is known about users’ perspectives and the limitations of these new automation techniques. We conducted an empirical study with 24 participants who generated 457 user stories using the US-Prompt technique.We assessed the quality of the user stories using the QUS framework, and we analyzed the acceptance of the technique using the TAM3 model. Results showed that the US-Prompt method was effective, with 87.5% of the stories meeting more than 75% of the quality criteria. Participants found the technique easy to use and useful, although they identified limitations such as formatting inconsistencies and concerns about reliability for critical tasks. This study thus offers a provocative reflection on the use of LLMs and points to new directions for future research in this emerging area.
Authors: Reine Santos, Igor Steinmacher, Tayana Conte, Ana Carolina Oran, Bruno Gadelha
Institution: Universidade Federal do Amazonas
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Awards: G2590611959, G29316538
OpenAlex ID: W7085582354