Work

Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise

Year: 2024

Type: conference-paper

Abstract: Most existing retrieval-augmented language models (LMs) assume a naïve dichotomy within a retrieved document set: queryrelevance and irrelevance.Our work investigates a more challenging scenario in which even the "relevant" documents may contain misleading or incorrect information, causing conflict among the retrieved documents and thereby negatively influencing model decisions as noise.We observe that existing LMs are highly brittle to the presence of conflicting information in both the fine-tuning and incontext few-shot learning scenarios.We propose approaches for handling knowledge conflicts among retrieved documents by explicitly fine-tuning a discriminator or prompting GPT-3.5 to elicit its discriminative capability.Our empirical results on open-domain QA show that these approaches significantly enhance model robustness.We also provide our findings on incorporating the fine-tuned discriminator's decision into the in-context learning process, proposing a way to exploit the benefits of two disparate learning schemes.Alongside our findings, we provide MACNOISE, a machine-generated, conflict-induced dataset to further encourage research in this direction 1 .

Source: Findings of the Association for Computational Linguistics: NAACL

DOI: 10.18653/v1/2024.findings-naacl.159

Authors: Giwon Hong, Jeonghwan Kim, Junmo Kang, Sung-Hyon Myaeng, Joyce Whang

Institutions: University of Edinburgh, University of Illinois Urbana-Champaign, Georgia Institute of Technology, Korea Advanced Institute of Science and Technology

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Ministry of Science and ICT, South Korea, Institute for Information and Communications Technology Promotion

Awards: G4700831490, G5198230845, G7537039438