Year: 2026
Type: conference-paper
Abstract: The adoption of large language models (LLMs) is transforming the peer review process, from assisting reviewers in writing detailed evaluations to generating entire reviews automatically.While these capabilities offer new opportunities, they also raise concerns about fairness and reliability.In this paper, we investigate bias in LLM-generated peer reviews through controlled interventions on author metadata, including affiliation, gender, seniority, and publication history.Our analysis consistently shows a strong affiliation bias favoring authors from highly ranked institutions.We also identify directional preferences associated with seniority and prior publication record, which can influence acceptance decisions for borderline papers.Gender effects are smaller but present in several models.Notably, implicit biases become more pronounced when examining token-level soft ratings, suggesting that alignment may mask but not fully eliminate underlying preferences.llm-review-bias
Institution: Helmholtz Center for Information Security
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
SDG: Reduced inequalities
Open Access status: hybrid
Funder: European Commission
OpenAlex ID: W4416259212