Work

WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics

Year: 2026

Type: conference-paper

Abstract: Web applications (web apps) have become a key arena for large language models (LLMs) to demonstrate their code generation capabilities and commercial potential.However, building a benchmark for LLM-generated web apps remains challenging due to the need for realworld user requirements, generalized evaluation metrics without relying on ground-truth implementations or test cases, and interpretable evaluation results.To address these challenges, we introduce WebCoderBench, the first realworld, generalized, and interpretable benchmark for web app generation.WebCoderBench comprises 1,572 real user requirements, covering diverse modalities and expression styles that reflect realistic user intentions.Web-CoderBench provides 24 fine-grained evaluation metrics across 9 perspectives, combining the rule-based and LLM-as-a-judge paradigms for fully automated, objective, and general evaluation.Moreover, WebCoderBench adopts human-preference-aligned weights over metrics to yield interpretable overall scores.Experiments across 12 representative LLMs and 2 LLM-based agents show that there exists no dominant model across all evaluation metrics, offering an opportunity for LLM developers to optimize their models in a targeted manner for a more powerful version.

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2026.acl-long.535

Authors: Chenxu Liu, Yingjie Fu, Wei Yang, Ying Zhang, Tao Xie

Institutions: Peking University, The University of Texas at Dallas

Language: English

Topic: Software Engineering Research

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid