How Many Papers Should Scientists Be Reviewing? An Analysis Using Verified Peer Review Reports

Abstract

The current peer review system is under stress from ever increasing numbers of publications, the proliferation of open-access journals and an apparent difficulty in obtaining high-quality reviews in due time. At its core, this issue may be caused by scientists insufficiently prioritising reviewing. Perhaps this low prioritisation is due to a lack of understanding on how many reviews need to be conducted by researchers to balance the peer review process. I obtained verified peer review data from 142 journals across 12 research fields, for a total of over 300,000 reviews and over 100,000 publications, to determine an estimate of the numbers of reviews required per publication per field. I then used this value in relation to the mean numbers of authors per publication per field to highlight a ‘review ratio’: the expected minimum number of publications an author in their field should review to balance their input (publications) into the peer review process. On average, 3.49 ± 1.45 (SD) reviews were required for each scientific publication, and the estimated review ratio across all fields was 0.74 ± 0.46 (SD) reviews per paper published per author. Since these are conservative estimates, I recommend scientists aim to conduct at least one review per publication they produce. This should ensure that the peer review system continues to function as intended.

Is a journal's ranking related to the reviewer's academic impact? (An empirical study based on Publons)

“…Considering the increasing difficulty in obtaining timely high‐quality reviews, Raoult (2020) proposed that scientists should conduct at least one review per publication they produced in order to ensure the peer review system continues to function. The median review to publication ratio of all the four groups of reviewers were less than 1, but each group had reviewers that had a ratio larger than 1.…”

“…This study selected the top 1,000 Publons partner journals that have received the largest number of verified reviews during a 12 months period (February 2019–February 2020) in the Research field of Medicine (Raoult, 2020). Journals were then selected that have an official Partnership with Publons to reduce bias.…”

“…Considering the increasing difficulty in obtaining timely highquality reviews, Raoult (2020) proposed that scientists should conduct at least one review per publication they produced in order to ensure the peer review system continues to function. Note: Data were expressed as median (interquartile ranges, IQR), comparison among the four groups were analysed using the Kruskal-Wallis test at significance level of p < 0.05, pairwise comparisons were conducted using the Nemenyi test at significance level of p < 0.05.…”

The Role of Publons in the Context of Open Peer Review

“…In that sense, Publons emerged as a platform to "reward" academics for their time and efforts, and as a form of public recognition. 6 However, such recognition can be perceived by some academics as a pennies-on-the-dollar scheme because even if demand for reviewers is high, the pool of available academics to complete the task is large, although this sink-and-demand dynamic might change if more academic journals enter the market, placing additional pressure on the same peer pool [20]. Pomponi et al [21] found that Publons' marketing approach was promotional, appealing to academics' vanity and sense of self-worth rather than on peer review-related novelty, also noting that most reviewers emerged from the US, India, then China.…”

“…Four Royal Society journals (Proceedings A, B, Royal Society Open Science and Open Biology) also continue to claim to be participating in the Publons TPR program and using OPR by publishing reviewers' reports, decision letters and responses alongside published articles. 20 The Royal Society encourages reviewers to disclose their names to authors, but this is a voluntary choice. Moreover, the publisher does not permit the publication of peer review information for rejected papers.…”