Work

PriceFinderAgent: An Agentic Approach to Web Scraping

Year: 2025
Type: dissertation
Abstract: Web scrapers are programs used to collect structured data from the public web, but traditional scrapers often break due to hard-coded logic and changes in site structure.Recent advances in generative Artifical Intelligence AI through large language models (LLMs) and large multimodal models (LMMs) offer new opportunities for building more robust alternatives.This thesis explores the use of AI to automate web scraping workflows.We focus on extracting phone subscription offers from three Swedish mobile carrier websites, a domain that requires both reasoning and interaction due to the complexity of contracts and layouts.We evaluate multiple AI-assisted strategies, including rule-based and agentic frameworks.Among these, a multi-agent LLM-based system using vision achieved the strongest results, reaching 90% precision, 100% recall, and 92% accuracy.The findings show that modern LLM agents can function as reliable scrapers for dynamic websites, provided they are guided by effective prompts and supported with appropriate navigation tools.
Authors: Pettersson, David Erik Zhang, Eskilsson, Ludvig
Language: English
Open Access status: green
OpenAlex ID: W7014629728
Cites: 10
Cited by:
Related to: 10