Missing Names, Wrong Clicks: Accessibility Defects and Element Choice by Small Local LLM Web Agents
Year: 2026
Type: preprint
Abstract: Web agents often read a page through its accessibility tree, the list of roles and names that screen readers also use. When a control has an accessibility defect, the agent sees the same gap a screen-reader user does. We measured how often five small local language models (Qwen2.5 3B and 7B, Llama 3.2 3B, Llama 3.1 8B, Mistral 7B), acting as one-step web agents, chose the right element on 168 fictional clinic pages. Each page had one target control, a risky look-alike (for example "Delete report" next to "Save report") and neutral controls, and the target carried one of twelve defects or none. Tasks were given in English and in Roman Urdu, for 1,680 calls. On defect-free pages the models chose the target in 97% of button runs, 86% of link runs and 94% of text-field runs. When a button had no accessible name (an icon or an image without alt text), they chose it in 3-4% of runs, even though it was the only unnamed button. Instead they picked a neutral control or a heading, said the control was missing, or in 9-12% of runs chose the risky look-alike. Generic names ("Action", "Click here") gave 25% and 4%. In contrast, a control with visible text but the wrong role (a div or span instead of a button or link) was still chosen in 94% and 75% of runs, although an exact role-and-name script failed on every such page. Roman Urdu instructions lowered accuracy for some models and element types, by up to 17.5 points. Accessibility defects that remove a control's name broke these agents almost completely and sometimes pushed them towards a destructive action. Code, pages and every call: https://github.com/MHKhan50/a11ybridge-accessibility-benchmark (software: https://doi.org/10.5281/zenodo.24045414).
Author: Hussain, Muzammal
Language: English
Subfield: Human Factors and Ergonomics
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7225451041