Work

A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems

Year: 2020

Type: conference-paper

Abstract: Most Natural Language Generation systems need to produce accurate texts.We propose a methodology for high-quality human evaluation of the accuracy of generated texts, which is intended to serve as a gold-standard for accuracy evaluations of data-to-text systems.We use our methodology to evaluate the accuracy of computer generated basketball summaries.We then show how our gold standard evaluation can be used to validate automated metrics.

Source: International Conference on Natural Language Generation

DOI: 10.18653/v1/2020.inlg-1.22

Authors: Craig Thomson, Ehud Reiter

Institution: University of Aberdeen

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: Engineering and Physical Sciences Research Council

OpenAlex ID: W3102266532