Open peer review: promoting transparency in open science

Abstract

Open peer review (OPR), where review reports and reviewers’ identities are published alongside the articles, represents one of the last aspects of the open science movement to be widely embraced, although its adoption has been growing since the turn of the century. This study provides the first comprehensive investigation of OPR adoption, its early adopters and the implementation approaches used. Current bibliographic databases do not systematically index OPR journals, nor do the OPR journals clearly state their policies on open identities and open reports. Using various methods, we identified 617 OPR journals that published at least one article with open identities or open reports as of 2019 and analyzed their wide-ranging implementations to derive emerging OPR practices. The findings suggest that: (1) there has been a steady growth in OPR adoption since 2001, when 38 journals initially adopted OPR, with more rapid growth since 2017; (2) OPR adoption is most prevalent in medical and scientific disciplines (79.9%); (3) five publishers are responsible for 81% of the identified OPR journals; (4) early adopter publishers have implemented OPR in different ways, resulting in different levels of transparency. Across the variations in OPR implementations, two important factors define the degree of transparency: open identities and open reports. Open identities may include reviewer names and affiliation as well as credentials; open reports may include timestamped review histories consisting of referee reports and author rebuttals or a letter from the editor integrating reviewers’ comments. When and where open reports can be accessed are also important factors indicating the OPR transparency level. Publishers of optional OPR journals should add metric data in their annual status reports.

Open Research Data and Open Peer Review: Perceptions of a Medical and Health Sciences Community in Greece

“…Regarding OPR and its potentials, the majority of the participants expressed their willingness to publish their papers' peer review reports and be reviewers in journals which follow OPR system, aligning with the content and results of other studies [57], [58]. The outcomes of the present research also reinforce the fact that OPR adoption is most prevalent in medical disciplines and agree with the steady growth in OPR adoption, which has been observed since 2001 [58].…”

“…Regarding OPR and its potentials, the majority of the participants expressed their willingness to publish their papers' peer review reports and be reviewers in journals which follow OPR system, aligning with the content and results of other studies [57], [58]. The outcomes of the present research also reinforce the fact that OPR adoption is most prevalent in medical disciplines and agree with the steady growth in OPR adoption, which has been observed since 2001 [58]. Even though the present study captured a positive trend of the medicine and health sciences community towards OPR and its potentials, it is quite questionable if the participants actually have realized its full potentials, which is also confirmed in Patel et al [59] study.…”

Mapping the Current Research Agenda on Scholarly Publishing: Scopus-Indexed Reviews

“…Some heated topics have emerged, with misinformed rhetoric preventing scholarly publishing from fast-track transformations. Whereas open access is the most visible part of Open Science, "efforts have extended to the availability of open data and software" (Wolfram, Wang, Hembree, & Park, 2020, p.1034.…”

“…Although it has been a professional and social filter and barrier to low-quality research for 300-plus years, peer review occasionally involves human failings and does not always prove the relevance and accuracy of the publication (Tennant et al, 2019). The research community put forward a lot of criticism and drawbacks of the peer review process, including bias, peer review fraud, and unfairness (Wolfram, Wang, Hembree, & Park, 2020); unnecessary delays (Benos et al, 2007); as well as a preponderance of incompetent reviewers, a lack of constructive criticism, and editorial passivity (Sciullo & Duncan, 2019).…”

“…Peer review aims to improve research and its reporting quality, reducing "the risk of publishing research that is sloppy, erroneous or, at worst, fabricated" (Wolfram, Wang, Hembree, & Park, 2020, p.1034.…”