Work

Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation

Year: 2023

Type: preprint

Abstract: Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics. In this work, we develop strong-performing automatic metrics for reference-based summarization evaluation, based on a two-stage evaluation pipeline that first extracts basic information units from one text sequence and then checks the extracted units in another sequence. The metrics we developed include two-stage metrics that can provide high interpretability at both the fine-grained unit level and summary level, and one-stage metrics that achieve a balance between efficiency and interpretability. We make the developed tools publicly available at https://github.com/Yale-LILY/AutoACU.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2303.03608

Authors: Liu, Yixin, Fabbri, Alexander R., Zhao, Yilun, Liu, Pengfei, Joty, Shafiq, Wu, Chien-Sheng, Xiong, Caiming, Radev, Dragomir

Institutions: Yale University, Salesforce (United States), Shanghai Jiao Tong University

Language: English

Topic: Advanced Text Analysis Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4323650592

Cites:

Cited by: 1

Related to: 10