Work

He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics

Year: 2021

Type: article

Abstract: We investigate how well BERT performs on predicting factuality in several existing English datasets, encompassing various linguistic constructions. Although BERT obtains a strong performance on most datasets, it does so by exploiting common surface patterns that correlate with certain factuality labels, and it fails on instances where pragmatic reasoning is necessary. Contrary to what the high performance suggests, we are still far from having a robust system for factuality prediction.

Source: Transactions of the Association for Computational Linguistics

DOI: 10.1162/tacl_a_00414

Authors: Nanjiang Jiang, Marie-Catherine de Marneffe

Institution: The Ohio State University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funder: National Science Foundation

Award: CAREER: Drawing inferences for human-like language understanding