Context-based User Story Enhancement with LLM and RAG : A Multi-agent System Implementation
Year: 2025
Type: dissertation
Abstract: User stories are central to agile development, yet their quality is often inconsistent, unclear, and misaligned with project goals.Existing improvement methods remain largely manual, error-prone, or limited to syntactic checks.This thesis presents a multi-agent system (MAS) that combines large language models (LLMs) with retrieval-augmented generation (RAG) to refine user stories, cluster them, and integrate with Jira for improved traceability.Developed with LangGraph and evaluated following a Design Science Research (DSR) approach, the system was tested on two datasets of ten user stories each.Domain experts scored quality across 11 criteria, showing improvements in clarity (2.6 → 4.1) and overall story quality, while inter-rater agreement indicated great reliability for grouping (α = 0.896) and modest agreement in linking (α = 0.466).This thesis contributes a novel MAS architecture for automated refinement and organization, an evaluation framework combining semantic quality assessment with agreement metrics, and empirical evidence on the potential and limits of AI-driven requirements management.Limitations include single-iteration evaluation and reliance on context-specific datasets, suggesting future research on multi-company validation and iterative feedback.This work advances the integration of AI and agile practices by offering a scalable, context-aware framework for enhancing user stories and backlog management.
Author: Nguyen, Khoa
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7132881445