What does better peer review look like? Definitions, essential areas, and recommendations for better practice

Abstract

Aim: To define a set of standards for better peer review. Method: We set out the expectations of five groups of stakeholders in the peer review process: authors, reviewers, editors, readers, and the general public. We then solicited case studies from people involved in peer review, to capture practical insights into how journal teams address the essential areas of integrity, ethics, fairness, usefulness and timeliness. Results: We received 40 case studies from stakeholders of journals published by Wiley in a range of subject areas from around the globe. The case studies identified areas of existing best practices and highlighted problems that might require better practices. Conclusion: We used this information to define essential areas of practice for peer review and to provide recommendations in each area, synthesizing these into a practical checklist to help journal teams improve their practices.

The state of the art in peer review

“…reviewer blinding versus identification) (Bastian 2017 ; Tennant 2017 ). Such diversity of views, while welcomed, has generally inhibited the development of any sort of standardisation behind the definition or the process (Ross-Hellauer 2017a ; Allen et al 2018 ), which, in turn, has created a fragmented landscape and makes any sort of comparative assessment into the efficacy of peer review problematic (Squazzoni, Grimaldo and Marušić 2017 ). This has been exacerbated by a general lack of coherence in the implementation of peer review, as well as a paucity of evidence and data sharing on it, making it difficult to draw rigorous conclusions about peer review systems.…”

Ten Hot Topics around Scholarly Publishing

“…But these subtleties are lost on the general public, who often only hear the statement that being published in a journal with peer review is the "gold standard" and can erroneously equate published research with the truth. Thus, more care must be taken over how peer review, and the results of peer reviewed research, are communicated to non-specialist audiences; particularly during a time in which a range of technical changes and a deeper appreciation of the complexities of peer review are emerging [48][49][50][51][52]. This will be needed as the scholarly publishing system has to confront wider issues such as retractions [39,53,54] and replication or reproducibility 'crises' [55][56][57].…”

“…For example, it reports a coverage of over 2000 journals in Asia ("230% more than the nearest competitor") 46 , which may seem impressive until you consider that in Indonesia alone there are more than 7000 journals listed on the government's Garuda portal 47 (of which more than 1300 are currently listed on DOAJ) 48 ; while at least 2500 Japanese journals listed on the J-Stage platform 49 . Similarly, Scopus claims to have about 700 journals listed from Latin America, in comparison with SciELO's 1285 active journal count 50 ; but that is just the tip of the iceberg judging by the 1300+ DOAJ-listed journals in Brazil alone 51 . Furthermore, the editorial boards of the journals contained in WoS and Scopus databases are dominated by researchers from western Europe and North America.…”

Ten myths around open scholarly publishing

“…But these subtleties are lost on the general public, who often only hear the myth that published in a journal with peer review is the "gold standard" and can erroneously equate published research with the truth. Thus, more care must be taken over how peer review, and the results of peer reviewed research, are communicated to non-specialist audiences; particularly during a time in which a range of technical changes and a deeper appreciation of the complexities of peer review are emerging (Bravo et al 2019;Tennant 2018;Squazzoni, Grimaldo, and Marušić 2017;Allen et al 2018). This will be needed as the scholarly publishing system has to confront wider issues such as retractions (Budd, Sievert, and Schultz 1998b;Fang and Casadevall 2011;Moylan and Kowalczuk 2016) and replication or reproducibility 'crisis' (Collaboration 2015;Munafò et al 2017a;Fanelli 2018).…”