Is Peer Review a Good Idea?

Abstract

Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. We conclude that on present evidence abolishing peer review weakly dominates the status quo.

Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment

“…They may also provide disincentives to beneficial behavior; for example, researchers may be disinclined to pursue publication of sound research if they believe that it is likely to be met with negative reviews (Cooper, DeNeve, & Charlton, 1997; Coursol & Wagner, 1986; Franco, Malhotra, & Simonovits, 2014). Given the available evidence, Heesen and Bright (2019) argued that abolishing prepublication peer review in its current form would have neutral or positive net value for the incentive structure in science and for individual researchers’ behavior.…”

The Cultivation of Social Work Knowledge: Towards a More Robust System of Peer Review

“…The interested reader may consult recent discussions in the literature for further information (Heesen & Bright, 2020;Helmer, Schottdorf, Neef, & Battaglia, 2017;Lee, Sugimoto, Zhang, & Cronin, 2013;Smith, 2006;). What I would like to do here, instead, is briefly describe three other limitations that are perhaps equally problematic (though not necessarily mutually exclusive); that peer review is: 1) unreliable, 2) that it does not prevent bad science from being published, and perhaps more speculatively, the claim that it 3) promotes groupthink and conservatism.…”

“…The greater the variability, (likely) the greater rate of error. This may be grounds for questioning the value of traditional peer review altogether (see Heesen & Bright, 2020 for an exploration of this question).…”

Scientific conclusions need not be accurate, justified, or believed by their authors

“…Even if we grant that the relationship between our expectations for the whole process and our expectations for individual inputs must be complex (see Mayo-Wilson et al, 2011), we evidently hold such individual contributions to some standards. The process of peer review as it now exists (on which see Heesen & Bright, 2020) seems to presuppose that there are conditions under which it would be inappropriate to bring a conclusion to the collective attention of one's field. Likewise, the very fact that many consider widespread replication failure (see e.g.…”