Year: 2026
Type: conference-paper
Abstract: Standard benchmarks for large language models (LLMs) assume that task feedback is truthful, but real-world reasoning often requires processing unreliable or adversarial information.We introduce WordleArenas, a benchmark platform that evaluates LLM reasoning robustness across a deception gradient.Building on Wordle and its deceptive variant Fibble (Chusap et al., 2025), we generalize to Fibble k (k = 0, . . ., 5 lies per row), creating a controlled evaluation of LLM robustness to misinformation.Across six arenas -standard Wordle (0 lies per row) through Fibble 5 (5 lies per row) -we evaluate 41 models from 10 providers across 3,749 games.We find that (1) even one lie per row causes catastrophic performance drops (average win rate falls from 41.1% to 18.7%), (2) a sharp deception cliff emerges at 2-3 lies where nearly all models collapse to ≤3% win rate, and (3) model robustness to deception is poorly predicted by standard benchmark rankings.A surprising Fibble 5 recovery emerges: some models recover partial performance when all feedback lies (average 9.5%), outperforming Fibble 3 (0.3%) and Fibble 4 (0.4%), because knowing that every tile lies restores deterministic -though partial -information.Our results demonstrate that truthful-feedback evaluations systematically overestimate LLM reasoning capabilities and that deception-aware benchmarks are essential for assessing real-world robustness.All code and data are publicly available.1
Author: Chang Liu
Institution: Ohio University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166832898