Year: 2025
Type: dissertation
Abstract: Large Language Models (LLMs) excel at processing natural language and unstructured data, driving significant advancements in automation. Traditional web automation approaches are often constrained by data variability and typically rely heavily on task-specific rules and extensive customization. In contrast, LLMs enable a more task-agnostic methodology, managing diverse tasks with minimal explicit instructions, thus facilitating more intuitive and flexible automation. This study introduces Robotic Process Automation Intelligence (RPAI), a component demonstrating these capabilities, designed to automate website interactions robustly and seamlessly through natural language. This study evaluates RPAI utilizing GPT-4o-mini across both the Mind2Web benchmark and real-world websites. RPAI achieved an overall step success rate of 52\% on Mind2Web dataset, with 71\% success rate when the correct element was succesfully included in parsed element list provided to the LLM. Subsequent enhancements in Document Object Model (DOM) parsing algorithm and improved step instructions led to noticeable improvements, achieving an overall success rate of 68\% for a selected subset of tasks. Despite these improvements, the Mind2Web benchmark posed challenges as an evaluation framework due to its inherent design, which differs significantly from the intended operational use of RPAI. To better assess RPAI’s practical performance, it was tested under more realistic conditions on five live, real-world websites with carefully curated step instructions. In this context, RPAI attained a 94\% step success rate, demonstrating substantial effectiveness for practical web automation tasks when applied in more suitable scenarios. While results underline RPAI’s potential to significantly streamline the development of web automation solutions, further research remains essential to enhance reliability and accuracy. Future improvements may include refining DOM parsing methods, enhancing mechanisms to mitigate false positives, advancing element-location strategies, and improving reliability, particularly in more complex and dynamic web environments.
Source: Aaltodoc (Aalto University)
Author: Nikkari, Daniel
Language: English
Subfield: Management Information Systems
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7061800942