Year: 2021
Type: dissertation
Abstract: Natural language inference (NLI) is the task of determining whether a "hypothesis" is true (entailment), false (contradiction), or undetermined (neutral) given a "premise".An inference contains two sentences, where the first sentence is considered the premise and the second the hypothesis.NLI models have been performing very well.However, recent studies have shown the performance of NLI models trained on the Stanford Natural Language Inference (SNLI) dataset are overestimated.Hypothesis-only classifiers can already label most of the inferences correctly without looking at the premise.This suggests that NLI models depend heavily on annotation artifacts present in the hypothesis.A bias seems to occur because certain annotation artifacts are often in relation to a specific label.This thesis will explore debiasing the SNLI dataset, which has this hypothesis-only bias.New examples will be generated to balance the annotation artifacts over the different labels.A few simple data augmentation techniques will be used to generate new premises in a way that the hypotheses will relate to 3 different labels, depending on the premise.The results show that the hypothesis-only bias in the augmented SNLI dataset has decreased and that DistilBERT, one of the high-performing NLI models, maintains similar performance after fine-tuning on this augmented dataset.Balancing the annotation artifacts over all labels is therefore an efficient way to mitigate hypothesis-only bias in a dataset.
Author: Albert Baas
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3191689059