Work

Hypothesis Only Baselines in Natural Language Inference

Year: 2018

Type: conference-paper

Abstract: We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI).Especially when an NLI dataset assumes inference is occurring based purely on the relationship between a context and a hypothesis, it follows that assessing entailment relations while ignoring the provided context is a degenerate solution.Yet, through experiments on ten distinct NLI datasets, we find that this approach, which we refer to as a hypothesis-only model, is able to significantly outperform a majorityclass baseline across a number of NLI datasets.Our analysis suggests that statistical irregularities may allow a model to perform NLI in some datasets beyond what should be achievable without access to the context.

Source: Joint Conference on Lexical and Computational Semantics

DOI: 10.18653/v1/s18-2023

Authors: Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, Benjamin Van Durme

Institutions: Johns Hopkins University, Birla Institute of Technology and Science, Pilani - Goa Campus, Birla Institute of Technology and Science, Pilani

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, Johns Hopkins University, Defense Advanced Research Projects Agency

OpenAlex ID: W2798358706