Work

A Dataset Generation Method for Bias Evaluation in Retrieval-Augmented Generation

Year: 2026

Type: conference-paper

Abstract: Retrieval-augmented generation (RAG) is a technique generates textual answers on separately retrieved information. While RAG reduces incorrect content in the answers, it has been shown to introduce and amplify biases in model outputs. There is still a lack of dedicated studies and benchmark datasets that systematically investigate how such bias amplification arises and propagates within the RAG pipeline. Drawing on prior work, this paper adopts a preference-based bias measurement framework and introduces a component-aware dataset construction method for datasets used for evaluating bias in real RAG pipelines, and instantiates it for occupation-gender bias using Wikipedia-based knowledge. Our goal is to share the dataset construction methodology, alleviate data scarcity in RAG bias research, and lay a foundation for future studies.

Source: Tampere University Institutional Repository (Tampere University)

Authors: Zhao, Yingqi, Efthymiou, Vasilis, Nummenmaa, Jyrki, Stefanidis, Kostas

Institutions: Tampere University, University of Tampere, Harokopio University of Athens

Language: English

Topic: Ethics and Social Impacts of AI

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Reduced inequalities

Open Access status: green

OpenAlex ID: W7164980648

FWCI:

Cites: 22

Cited by:

Related to: 10