A study of reviewing at the International Journal of Osteoarchaeology: Accepting, declining, fairness, and responsibilities
Year: 2023
Type: article
Abstract: Before I became a journal editor, I always saw the peer review process as an intrinsic part of academic life (Burnham, 1990). While peer review is certainly not a perfect system, and a number of its limitations are well known, it appears to be the most impartial and independent method currently available to assess the quality of research submitted to a journal (Bazi, 2020; Tennant & Ross-Hellauer, 2020). If I wanted to submit my research papers to journals where others would have to review them, then it only seemed fair that I review the submissions of others in return. So long as papers I was asked to review fell within my area of expertise, it never occurred to me to decline. Even if my life was going through a busy patch, I made time to have a look at the paper at some point in the following few weeks and send in my thoughts. When I took over the role as editor for human remains at the International Journal of Osteoarchaeology, I did my best to carefully focus my invitations to appropriate reviewers. My aim was to try to ensure only those who had published on that topic before were invited. I also tried to avoid asking a reviewer too many times in 1 year, to share the load fairly. As past research has shown that an invitation e-mail with wording individualized to that reviewer also increases the likelihood of a positive response to a review invitation (Franceschet et al., 2022), we reworded the automated journal invitations to be more friendly and less formal. However, despite this, it has become clear that many academics do not have the same approach to reviewing as do I. It is really very difficult to find suitable experts to agree to review many of our submitted papers. As authors who have submitted their paper will be well-read on that field, having researched past publications in the topic, we ask that those submitting papers propose suitable independent reviewers to help the editor ensure the most appropriate experts are invited. This is a common practice in the significant majority of scientific journals (Liang, 2018), although those proposed by the authors do clearly need to be independent and not chosen in order to give favorable reviews (Hausmann et al., 2018). However, it is clear that quite a few authors do not enter any proposed reviewers, and some suggest reviewers that seem to have no expertise whatsoever in the topic of their submitted paper. As an editor, I am regularly baffled as to why those submitting a paper choose to make their own lives harder by either not making the effort to read the journal author guidelines, or not making the effort to follow them. Past research has also noted that papers for whom journal editors struggle to find reviewers are more likely to be rejected than papers that are promptly accepted by potential reviewers (Fox, 2017). Clearly, it is in author's best interests to propose a good selection of relevant potential reviewers when they submit, to ensure journal editors invite the most relevant experts. In order to quantify this peer review challenge, I undertook a 12 month prospective study of the outcome of IJO reviewer invitations, from September 2021 to August 2022. Over this time, 66 papers on human osteoarchaeology were sent out for peer review. Broken down by field, 25 were on paleopathology, 19 were on human anatomy or age and sex estimation, nine were on isotope analysis, seven on the skeletal consequences of function and behavior, five on burial archaeology, and one on the curation of skeletal remains in museums. For these 66 papers, only 53 authors submitted the details of reviewers they proposed as suitable experts in their field. This means the authors of 20% of papers decided to not spend the few minutes it takes to enter the names, university, and e-mail address for even two or three such experts. In the process of peer review of these 66 manuscripts, a total of 300 invitations to reviewers were sent out. Of the 300 invitations, 156 declined to review the paper (52%). The mean number of reviewers invited per paper was 4.55, and the mean number of reviews received per paper was 1.94. One paper required eight reviewer invitations before two reviews were received, and two papers required nine reviewer invitations to do so. If a reviewer declines an invitation to review, they are asked to suggest alternative experts in their field who would be appropriate for us to invite in their place. However, of the 156 reviewers who declined, only 51 suggested alternatives. That means 67% of those invited to review a paper, and who declined to do so, did not suggest alternative colleagues. So how does the IJO experience compare with other scientific journals? The literature would suggest that virtually all journal editors struggle to find reviewers with the right expertise for each submitted paper. The proportion of academics who decline to take on reviews is frequently in the range of 30% to 60% (Albert et al., 2016; Djupe, 2015; Fox, 2017; Franceschet et al., 2022; Willis, 2016). There is also some evidence for a declining trend in acceptance rates, with one multijournal study showing 56% of reviewer invitations in 2003 generating a review, compared with just 37% in 2015 (Fox et al., 2017). The vast majority of those declining a reviewer invitation do not state their decision is due to a conflict of interest, or that the paper was not in their field of expertise, but rather they state the reason was that they do not have the time to undertake the review (Willis, 2016). Studies have investigated whether payments to reviewers or nonmonetary rewards such as certificates help, but found no meaningful increase in the likelihood of taking on a review (Gasparyan et al., 2015; Zaharie & Seeber, 2018). Gender differences in the peer review process have been noted in past research, with many journals having women underrepresented on their editorial boards (Fox et al., 2016, 2019). Since refreshing the IJO editorial board earlier this year, the sex ratio of the editors and editorial board is 57% female to 43% male. Therefore, we feel we are ahead of the curve in gender equality at the IJO. We at the IJO would like to thank all our helpful colleagues in the field who review papers submitted to the journal. Being asked to review is an indicator of the esteem in which we hold you and your work. We know that academic life is a busy one, with many pressures and deadlines to meet. If the following 2–3 weeks will be taken up by marking student coursework or creating a grant proposal, then rather than declining the invitation, a quick e-mail to an editor letting them know the date by which you could do the review is a constructive way forward. It is clearly right that someone who is physically ill, or struggling with stress, anxiety, or other mental health problems, should explain to an editor that they are not in a position to undertake reviews at that time. For the rest of us it seems fair that if we submit papers to research journals, then we should agree to review at least two times as many papers ourselves. The figure of two times as many is needed to ensure at least two opinions are given on every submitted paper. If for any reason taking on a review is just not possible, then your suggestions for alternative reviewers are always appreciated. This has the potential benefit to introduce editors to emerging members of the field and so broaden the range of individuals in the reviewing pool and improve representation across gender and geographic range. If we can all work together on this challenge, then the next time you submit a paper to the journal, there will be a swifter path through the reviewing process. There will also be a greater likelihood that your paper is assessed by colleagues with the best match of skills and expertise, who can give you feedback that you value and respect (Stam et al., 2022). The author has no conflict of interest to declare. The data that support the findings of this study are available from the corresponding author upon reasonable request.
DOI: 10.1002/oa.3184
Author: Piers D. Mitchell
Institution: University of Cambridge
Language: English
Subfield: History and Philosophy of Science
Field: Arts and Humanities
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W4320018201