Year: 2020
Type: conference-paper
Abstract: High quality data forms the bedrock for building meaningful statistical models in NLP.Consequently, data quality must be evaluated either during dataset construction or post hoc.Almost all popular summarization datasets are drawn from natural sources and do not come with inherent quality assurance guarantees.In spite of this, data quality has gone largely unquestioned for many recent summarization datasets.We perform the first large-scale evaluation of summarization datasets by introducing 5 intrinsic metrics and applying them to 10 popular datasets.We find that data usage in recent summarization research is sometimes inconsistent with the underlying properties of the datasets employed.Further, we discover that our metrics can serve the additional purpose of being inexpensive heuristics for detecting generically low quality examples.
Authors: Rishi Bommasani, Claire Cardie
Institutions: Stanford University, Cornell University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W3099286868
