Evaluating and Mitigating Inherent Linguistic Bias of African American English through Inference
Year: 2022
Type: other
Abstract: Recent studies show that NLP models trained on standard English texts tend to produce biased outcomes against underrepresented English varieties. In this work, we conduct a pioneering study of the English variety use of African American English (AAE) in NLI task. First, we propose CODESWITCH, a greedy unidirectional morphosyntactically-informed rule-based translation method for data augmentation. Next, we use CODESWITCH to present a preliminary study to determine if demographic language features do in fact influence models to produce false predictions. Then, we conduct experiments on two popular datasets and propose two simple, yet effective and generalizable debiasing methods. Our findings show that NLI models (e.g. BERT) trained under our proposed frameworks outperform traditional large language models while maintaining or even improving the prediction performance. In addition, we intend to release CODESWITCH, in hopes of promoting dialectal language diversity in training data to both reduce the discriminatory societal impacts and improve model robustness of downstream NLP tasks.
Source: Underline Science Inc.
DOI: 10.48448/bd66-qt03
Authors: International Committee for Computational Linguistics 2022, Dacon, Jamell, Dacon, Jamell, Liu, Haochen, Liu, Haochen, Tang, Jiliang, Tang, Jiliang
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
SDG: Reduced inequalities
Open Access status: green
OpenAlex ID: W6948162300