Cost-Efficient Observation for LLM-Based UI Grounding: Measuring Predicate-Relative Attribute Sufficiency Within a Symbolic Representation
Year: 2026
Type: article
Abstract: A large language model (LLM)-based agent uses text observations to find UI elements. Their content affects token cost and grounding success. Observation-reduction work asks how much to remove; we ask which attribute subsets are sufficient under a fixed success predicate. We compare 6000 events across 24 synthetic pages, varying attributes and serialisations. Removing volatile class and id values increases success by 9.48 pp (95% CI [+4.90, +15.52] pp). The secondary metric P_now (unique correct match) changes little where a stable attribute exists but decreases otherwise. The improvement comes from removing unstable locator candidates, not adding information. Abbreviating attribute names reduces success by 12.08 pp, saving 0.2% of tokens. Removing JSON syntax while keeping attribute names reduces cost without a significant change in success. The cost-optimal profile (COP), best among 21 main configurations, achieves 10.00 pp higher success with 34.0% fewer tokens than full-attribute JSON. In a one-attempt comparison, the accessibility snapshot uses half the tokens but performs worse; neither dominates. The direction persists across three gpt-4.1 generations and four models under non-identical decoding settings (for o4-mini, via the COP). A 33-site archived-page pilot shows a smaller gain in that direction. Results concern predicate-relative sufficiency in HTML DOM with CSS locators.
Source: Applied Sciences
DOI: 10.3390/app162211180
Authors: Anton Angelov, Milena Lazarova
Institution: Technical University of Sofia
Language: English
Topic: Web Data Mining and Analysis
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W7225803659