Do AI-Generated Pull Requests Get Rejected More? (Yes but Why?)
Year: 2026
Type: conference-paper
Abstract: This paper investigates an important question in the era where AI can autonomously generate code and create pull requests: do AI-generated pull requests get rejected more than human-written ones and why do they get rejected? This investigation will help us understand how AI can support the open source community and its limitations. We conduct an in-depth examination of 1,779 rejected PRs (594 AI-generated and 1,185 human-authored), supported by 19,450 review comments and 28,875 review events. We further construct a ten-category taxonomy of rejection reasons grounded in reviewers’ explanations and code-level evidence. Our comparative analysis reveals similarity and differences in failure patterns: AI-generated PRs are more likely to be rejected due to incomplete implementation issues and testing-inadequate issues, human PRs showed a highly similar composition of rejection categories. However, the proportion of the other category is higher for human PRs than for AI PRs. This suggests that rejections of human PRs more often involve heterogeneous or context-specific reasons that do not fall neatly into predefined categories, whereas AI-generated PRs tend to fail in more repetitive and well-defined ways. Failure modes in AI-generated PRs are also more consistent and predictable, with description-level features providing stronger predictive signals compared to human-authored PRs. These findings provide empirical insights for improving AI coding assistants and open new opportunities for designing automated mechanisms and PR-review support tools tailored to AI-generated code.
Authors: Yiru Wang, Zhou Yang
Institutions: University of Alberta, Alberta Machine Intelligence Institute
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7172011159