Year: 2025
Type: dissertation
Abstract: Applications, especially ones operating on the web, are very often deprecated due to fast moving technology. As up-keeping applications are costly and time-consuming continuous projects, many applications are left behind by their up keepers, causing challenges in security, reliability, and compatibility with other systems. Therefore, a fast and reliable system is needed to automate the needed updates for the applications. This thesis extends a previous study of a Large Language Model (LLM)-based multi-agent pipeline system now introduced as Verifying Agent Pipeline Updater (VAPU) designed to refactor and modernise code. The thesis aims to find how different LLMs, the temperature used, and different tasks impact the performance of VAPU. Furthermore, this thesis investigates the challenges and opportunities of using a multi-agent system for updating code to provide understanding of multi-agent systems use in the practical sense. A web legacy application was used as a case study to evaluate VAPU’s performance. Six CakePHP view files from the legacy application were updated in five different LLMs at temperatures 0 and 1 with and without VAPU. VAPU’s performance was further verified with 20 GitHub Python projects in different feature extension and modernisation tasks with all models, and both methods. The challenges and benefits of the multi-agent system’s functionality were observed during the testing. Comparison with different LLMs showed differences in performance. Medium-sized LLMs exceeded in smaller version update tasks, while higher-parameter models were better in more difficult tasks. With the temperature lowered to 0, VAPU improved with a reduction in errors and a higher probability of completing the requirements. Based on the evaluation of the Python files and depending on the LLM used, VAPU can achieve an improvement up to 22.5% in various code update tasks. The improvements were focused on cases with complex code. This thesis demonstrates that a multi-agent system is capable of updating code with a possibility of higher rate of completed requirements compared to a prompt for an LLM. The thesis contributes additionally showing the benefits of lower temperature in code updating multi-agent systems. The benefits and challenges observed during the testing provide practitioners with opportunities to further improve similar systems further for updating code. VAPU provides a working foundation for practitioners for code updating multi-agent systems. The code for VAPU is publicly available at https://github.com/alasalm1/Multi-agent-pipeline
Author: Ala-Salmi, Valtteri
Institutions: Tampere University, University of Tampere
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7132990210