Self-Healing Web Data Extraction Pipelines: Combining Lightweight LLMs and Deterministic Parsers for E-Commerce and Telecom Intelligence
Year: 2026
Type: article
Abstract: Production web data extraction pipelines for e-commerce and telecommunications intelligence face a persistent operational trade-off: deterministic parsers (XPath/CSS via lxml and parsel) offer near-zero marginal cost and millisecond latency but break frequently under frontend DOM mutations, whereas routing raw HTML through Large Language Models (LLMs) incurs prohibitive token costs, high latency, and exposure to indirect prompt injection. In this technical note (Report ID: UNAL-CS-2026-TN02), we present a self-healing extraction architecture deployed on serverless cloud workers that decouples routine parsing from LLM-assisted recovery. Under normal operation, extraction relies strictly on deterministic selectors validated against strict Pydantic schemas. When structural drift or schema violations trigger an anomaly circuit breaker, the system isolates the mutated HTML region via Lowest Common Ancestor (LCA) DOM pruning, encapsulates the untrusted snippet within cryptographic XML nonce boundaries to mitigate indirect prompt injection, and invokes a lightweight LLM out-of-band solely to synthesize a patched XPath/CSS selector. Once verified against historical schema invariants in a sandbox, the new selector hot-swaps into the production registry. Evaluated across Latin American and Colombian retail and telecommunications portals, this hybrid pipeline resolves layout breaks autonomously while reducing token consumption by over 98% compared to full-page LLM scraping.
Author: Urrego Lancheros, Julian David
Institution: Universidad Nacional de Colombia
Language: English
Topic: Web Data Mining and Analysis
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225625085
FWCI:
Cites:
Cited by:
Related to: