EVBench: A Comprehensive Benchmark of Video Generative Models for E-commerce Scenarios
Year: 2026
Type: article
Abstract: Assessing the quality of videos generated by video generation models plays an important role in improving the capability of these models. Existing video generation benchmarks like VBench and EvalCrafter mainly focus on general aspects of video quality and video–text consistency, but they could be ineffective for the unique requirements of e-commerce video generation, such as the consistency of fine-grained product categories and details, the usage of products, camera movements, and so on. To solve this issue, we introduce EVBench , a comprehensive E -commerce V ideo Bench mark to evaluate the capability of video generation models for e-commerce scenarios. Our EVBench contains two parts, i.e., EVBench-T2V and EVBench-I2V, to evaluate the text-to-video models and image-to-video models, respectively. EVBench-T2V contains 300 structured prompts covering 300 fine-grained product categories and various backgrounds, while EVBench-I2V has 120 image–prompt pairs of 120 categories with diverse instructions like camera movements and usage scenarios. Moreover, a set of novel evaluation metrics is designed to quantitatively measure the capability of video generation models from 10 dimensions. Furthermore, we conduct human preference annotation to validate the alignment between our benchmark and human perception. Finally, based on EVBench, we comprehensively evaluate 15 mainstream open source and closed-source models to discover their capabilities and provide valuable insights for e-commerce video generation.
DOI: 10.1145/3830242
Authors: Haofeng Chen, Jinkai Zheng, Kun Liu, Xinchen Liu, Liangqiong Qu, Ming Li, Wu Liu, Chenggang Yan
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7171494809