Work

Reputation or peer review? The role of outliers

Year: 2018

Type: article

Abstract: We present an agent-based model of paper publication and consumption that allows to study the effect of two different evaluation mechanisms, peer review and reputation, on the quality of the manuscripts accessed by a scientific community. The model was empirically calibrated on two data sets, mono- and multi-disciplinary. Our results point out that disciplinary settings differ in the rapidity with which they deal with extreme events-papers that have an extremely high quality, that we call outliers. In the mono-disciplinary case, reputation is better than traditional peer review to optimize the quality of papers read by researchers. In the multi-disciplinary case, if the quality landscape is relatively flat, a reputation system also performs better. In the presence of outliers, peer review is more effective. Our simulation suggests that a reputation system could perform better than peer review as a scientific information filter for quality except when research is multi-disciplinary and in a field where outliers exist.

Source: Scientometrics

DOI: 10.1007/s11192-018-2826-3

Authors: Francisco Grimaldo, Mario Paolucci, Jordi Sabater-Mir

Institutions: Universitat de València, National Research Council, Institute of Cognitive Sciences and Technologies, Consejo Superior de Investigaciones Científicas, Artificial Intelligence Research Institute

Language: English

Topic: scientometrics and bibliometrics research

Subfield: Statistics, Probability and Uncertainty

Field: Decision Sciences

Domain: Social Sciences

Open Access status: hybrid

Estimated APC paid: $3,190

Funders: Universitat de València, Ministerio de Ciencia e Innovación