Year: 2026
Type: conference-paper
Abstract: Large language models (LLMs) are increasingly deployed as web-browsing agents, yet current benchmarks mostly evaluate explicit queries over clean factual evidence.We introduce MelonBench, a 665-item benchmark for coded entity resolution over noisy openweb user-generated content, instantiated with mid-to-long-tail public figures in Chinese entertainment blind-item discourse.Solving MelonBench requires agents to decode obfuscated clues, retrieve fragmented evidence, and identify the target entity.The benchmark is constructed through a five-stage semi-automatic pipeline with atomic event stratification, adversarial faithfulness auditing, and expert verification.Across 30 systems, the best non-search model reaches 25.1% accuracy and the best search-enabled system reaches 49.9%.Controlled comparisons of the same models with and without web search confirm consistent gains from web search, but also substantial remaining headroom.Diagnostic experiments identify semantic de-obfuscation and openended entity recall as key bottlenecks, and show that reasoning-centric search can outperform report-oriented agents.An 80-item finance pilot provides preliminary evidence of transfer beyond entertainment within a Chinese-language setting.MelonBench offers a challenging initial testbed for search agents under intentional obfuscation and noisy evidence.
Language: English
Topic: Data Quality and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W7224646972