The use of rating and Likert scales in Natural Language Generation human evaluation tasks: A review and some recommendations
Year: 2019
Type: conference-paper
Abstract: Rating and Likert scales are widely used in evaluation experiments to measure the quality of Natural Language Generation (NLG) systems.We review the use of rating and Likert scales for NLG evaluation tasks published in NLG specialized conferences over the last ten years (135 papers in total).Our analysis brings to light a number of deviations from good practice in their use.We conclude with some recommendations about the use of such scales.Our aim is to encourage the appropriate use of evaluation methodologies in the NLG community.
DOI: 10.18653/v1/w19-8648
Authors: Jacopo Amidei, Paul Piwek, Alistair Willis
Institution: The Open University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W2990527403
