Work

An Analysis of Natural Language Inference Benchmarks through the Lens of Negation

Year: 2020

Type: conference-paper

Abstract: Negation is underrepresented in existing natural language inference benchmarks.Additionally, one can often ignore the few negations in existing benchmarks and still make the right inference judgments.In this paper, we present a new benchmark for natural language inference in which negation plays an important role.We also show that state-of-the-art transformers struggle making inference judgments with the new pairs. Original pairNew pair w/ negation RTE T: Tropical Storm Debby is blamed for several deaths across the Caribbean.T neg : Tropical Storm Debby is not blamed for several deaths across the Caribbean.H: A tropical storm has caused loss of life.H neg : A tropical storm has not caused loss of life.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/v1/2020.emnlp-main.732

Authors: Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei, Eduardo Blanco

Institutions: University of North Texas, Universitat de Barcelona

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, Nvidia

Award: CAREER: Understanding Negation in Positive Terms