Year: 2022
Type: preprint
Abstract: We present a benchmark dataset for evaluating method-level code generation task. The benchmark contains a dataset of 175 samples for automated evaluation and a dataset of 161 samples for manual evaluation. We also present a new metric for automatically evaluating the correctness of the generated code, and a set of criteria to manually evaluating the overall quality of the generated code.
Source: arXiv (Cornell University)
Authors: Hao, Yiyang, Li, Ge, Liu, Yongqiang, Miao, Xiaowei, Zong, He, Jiang, Siyuan, Liu, Yang, Wei, He
Institution: Peking University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6901758419