Peer review and secrecy in the “Information Age”

Abstract

The subject of peer review has come under increased scrutiny partially due to high profile cases of fraud and other misinformation appearing in academic journals. At the same time, new publishing technologies have made it possible to experiment with new forms of peer review, while the "Open Archives Initiative" has the potential to allow authors to bypass the traditional peer review process entirely. In this paper, I examine some rationales given for the need for secrecy in peer review, especially as it relates to the perceived need for anonymity on the part of reviewers and sometimes even authors. I will propose a basic framework for secrecy and transparency as it relates to the peer review process of academic journals. IntroductionIn December 2006, there was a debate about peer review featured prominently on the website of the journal Nature. In addition to a series of articles discussing various systems of peer review; its effect on the quality and value of research; research ethics; new technical implementations of peer review; and a series of individual perspectives and reflections on the subject; it includes an experiment in "open" peer review. Authors were given the option of posting their manuscripts for open peer review at the same time they submitted their work for the more traditional, confidential peer review process. For authors who elected to participate in the experiment in open peer review, comments on their research were solicited from qualified members of the public. Those who wished to comment could do so on the moderated forum on the Nature website. Unlike traditional peer review, anonymous comments were not allowed. The self-selected reviewers were required to give both their name and institutional affiliation. The stated purpose of this experiment was to "measure the level of participation among authors and the quality of comments received by members of the specialist community who are not the selected peer reviewers of the manuscripts concerned." (Nature website, 2006).Despite some initial enthusiasm, the four month experiment met with mixed results. Only five percent of authors chose to post their papers on the open server for comments, perhaps due to the fear of being "scooped". Except for a handful of relative highly commented on papers, there was a dearth of substantive comments despite statistics that showed a high level of web traffic.Biology Direct is an example of a journal that has gone beyond the experimentation phase, and formally implemented a system of open peer review. An online journal that began publication in January 2006, the editorial policy requires that reviewer comments, along with the author's response, are published alongside the article. The main hurdle to authors to overcome in order to qualify for publication is that three members of the editorial board must become sufficiently interested in the research to agree to act as reviewers or to solicit outside reviewers. Once this requirement is met, the policy even allows for publication of papers that have r...

Double-blind review in software engineering venues

“…For example, parallel to the debate on the anonymity of authors is the discussion on the opportunity to disclose reviewers' identities [21]. In fact, there have been questions regarding the bias, negligence, and self-interest of reviewers [22], [23] that may be intensified by their anonymity [2]. On the other hand, the anonymity of reviewers is believed to increase their frankness, therefore the quality of their reviews [2], and to reduce cases of open rivalry.…”

“…Peer review is the practice that scientists use to evaluate research manuscripts and artifacts. The outcome of peer review is strongly connected to the advancement of scientific knowledge, researchers' careers [1], and funding decisions from governments and businesses [2].…”

“…In past decades, the peer review process has increasingly been called into question [2]. Researchers provided empirical evidence on the shortcomings of peer review, for example, related to reviewers' biases [3], low agreement among reviewers [4], and scarce fit to identify impactful ideas [5]- [7].…”

“…One of the most recurring topics of debate on how to affect the quality of reviews concerns the visibility of those involved [2]. In particular, whether authors' identity should be visible to the anonymous reviewers, i.e., single-blind review (SBR), or not, i.e., double-blind review (DBR).…”

Peer review: A good but flawed system?

“…The Journal of Child Health Care uses a double-blind review system. The anonymity afforded by double-blind review is said to encourage the reviewers' candour, impartiality and freedom of expression (Weicher, 2008), although it is also possible for candour to go too far and reviewers can be disrespectful while hiding behind their anonymity. One of the core roles of the editor is to work collegiately with the reviewers and the authors to ensure that authors receive constructive, clear and helpful comments about their papers.…”