Work

Paraphrasing for automatic evaluation

Year: 2006

Type: conference-paper

Abstract: This paper studies the impact of paraphrases on the accuracy of automatic evaluation. Given a reference sentence and a machine-generated sentence, we seek to find a paraphrase of the reference sentence that is closer in wording to the machine output than the original reference. We apply our paraphrasing method in the context of machine translation evaluation. Our experiments show that the use of a paraphrased synthetic reference refines the accuracy of automatic evaluation. We also found a strong connection between the quality of automatic paraphrases as judged by humans and their contribution to automatic evaluation.

DOI: 10.3115/1220835.1220893

Authors: David Kauchak, Regina Barzilay

Institutions: University of California San Diego, MIT Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Funders: National Science Foundation, Defense Advanced Research Projects Agency

Awards: G3190074090, G6671297155, CAREER: Content and Cohesion Models, with Applications to Text Summarization and Natural Language Generation

OpenAlex ID: W2132019450

FWCI: 10.54

Cites: 21

Cited by: 222

Related to: 10