Year: 2026
Type: dissertation
Abstract: With the rapid advancement of Large Language Models (LLMs), there is growing interest in leveraging these models to automate software engineering tasks such as code refactoring. Code refactoring aims to improve the internal quality of software systems without altering their external behavior, making it a critical activity for long-term maintainability. While prior work has demonstrated the potential of LLMs to assist with refactoring, a comprehensive understanding of their effectiveness and limitations in realistic development settings remains underexplored. This gap is important because unreliable automated refactoring can introduce subtle behavioral regressions, increase debugging effort, and reduce developer trust in AI-assisted software engineering tools. Understanding when LLM-based refactoring succeeds or fails is therefore essential for safely integrating these models into practical development workflows. In this thesis, we present an empirical investigation of LLM-driven code refactoring and propose methods to improve its reliability. We first evaluate the refactoring capabilities of five state-of-the-art LLMs, StarCoder2, LLaMA 3, GPT-4o mini, GPT-4o, and DeepSeek-v3, across 30 open-source Java projects. We assess their ability to improve static code quality, reduce code smells, and preserve functional correctness through unit test execution. Our results show that closed-source production models, such as GPT-4o and DeepSeek-v3, achieve unit test pass rates exceeding 90% in multi-file refactoring scenarios, while LLaMA 3 achieves the highest reduction in code smells. We further demonstrate that prompt engineering techniques, such as chain-of-thought and one-shot prompting, can enhance both correctness and diversity of refactorings. Despite these results, we find that LLM-generated refactorings frequently introduce functional regressions. To address this limitation, we analyze failure cases, identifying key causes such as context misunderstanding, inconsistent renaming, and incomplete code generation. Building on these insights, we design an automated, test-guided repair framework that incorporates static analysis, unit test feedback, explanation agents, and project-level context to iteratively correct faulty refactorings. Our strongest configuration repairs 70.8% of the failures remaining after static repair, increasing the cumulative unit test pass rate to 94.2%. Overall, this thesis provides empirical evidence and practical repair strategies that support the development of more reliable LLM-based refactoring assistants for real-world software maintenance.
Author: Cordeiro, Jonathan
Institution: Queen's University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7209116714