Work

Problems with Peer Review Shine a Light on Gaps in Scientific Training

Year: 2023

Type: editorial

Abstract: T he term "peer review" often brings to mind the process by which scientific journals solicit experts to evaluate manuscripts prior to their publication.Scientists have debated the utility and value of peer review since its modern deployment in the 1970s (1).Problems with peer review and concerns about its role in scientific gatekeeping have led some to propose, in scientific manuscripts (2, 3) and in media articles (4), eliminating peer review altogether.One major concern is that peer review has done little to identify failures of scientific rigor (e.g., improper statistics, missing controls, data fabrication, or manipulation, detection of bias) (5, 6).Another issue is that it has revealed institutional, geographic, racial, and gender bias (7, 8).To address these problems, recent innovations in peer review include blinding, double blinding, open peer review, and "community peer review" in the form of open, public feedback on preprint servers and on other scientific forums.Taking a step back, we wonder whether current problems with peer review point to gaps in scientific training.Whether or not scientists continue to perform peer review as a service to scientific journals, we must still be able to identify major errors in scientific manuscripts, but current evidence suggests that we cannot.While post-publication review by the scientific community can eventually right publishing wrongs or present corrections to flawed studies, this can be a lengthy process that consumes time, resources, and funding (9).Rather than eliminate peer review, we propose that we use the process of peer review as an instrument to train scientists on how to evaluate science critically, fairly, and civilly through the development of formal peer review training programs.Unfortunately, this kind of training program is not widely implemented at any level of scientific training.What problems with scientific training has peer review illuminated?In our current system, many scientists gain experience with peer review by "learning on the job" when invited to review by an editor or as apprentices contributing to reviews accepted by mentors.The lack of formal training may underlie the expectation gap between what biomedical editors report wanting to see in peer reviews and what referees deliver.Editorsoften leading scientists in the fieldidentify core problems with submitted peer reviews, including unacceptable text or tone, requests for additional experiments outside the scope or peer review guidelines of the journal, and misuse of the confidential notes to editors (2, 10).Additionally, although journal editors currently have the responsibility and power of removing inappropriate comments from peer reviews, receiving scathing reviews is unfortunately a common experience.Uncivil peer review comments, in turn, disproportionately harm early career researchers and scientists from historically excluded and marginalized communities (11).Closing the gap between the goals of peer review and reality may also be hindered by a paucity of objective scoring rubrics for editors to rate review quality.Reviews are frequently evaluated subjectively, yet there is only modest agreement among editorial

Source: mBio

DOI: 10.1128/mbio.03183-22

Authors: Diana M. Proctor, Nsa Dada, Anna Serquiña, Julia L. E. Willett