Year: 2026
Type: preprint
Abstract: The Reviewer 2 stereotype conflates recommendation severity, polarization, disagreement, and editorial influence. We examined these dimensions using metadata-only records reconstructed from the Open Review-Based (ORB) archive, focusing on ordinal rating behavior rather than review-text tone or civility. After deduplication, the database contained 62,935 manuscripts and 157,070 review records. The primary explicit-label sample included 8,885 manuscripts and 29,166 reviews, concentrated predominantly in OpenReview machine-learning conferences (7,966 from ICLR 2018–2021). Manuscript-balanced ordinal contrasts, venue-specific mixed models, endpoint analyses, and independent editorial outcomes separated observable positions from decision mechanisms. Label two showed no clear excess harshness: -0.008 (95% CI -0.022 to 0.006). Its excess probability of either scale endpoint was 0.003 (95% CI -0.004 to 0.009); median-distance and broader-tail analyses likewise did not support general polarization. Public labels matched creation chronology on only 30.1% of eligible reviews. Among 5,964 decision manuscripts, greater second-reviewer severity was associated with rejection, but the severity-by-peer-disagreement interaction was inconclusive (P = 0.637). In a retrospective audit of 1,180 held-out final archived snapshots, an equal-weight mean outperformed a separate-R2 model in Brier loss (0.0830 vs. 0.0853). A paired MIDL subset showed reduced disagreement after rebuttal across 85 manuscripts, without clear second-reviewer-specific change. The supplied small F1000Research case was treated solely as an unverified historical audit, not an independent replication. Within these observed open-conference settings, the evidence favors a measurement- and institution-dependent account over universal Reviewer 2 ordinal harshness or polarization; generalizability to traditional cross-disciplinary journals, linguistic hostility, and prospective editorial weighting remain unmeasured.
Source: Research Square
Institution: IPB University
Language: English
Subfield: Library and Information Sciences
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7234985481
