Does incentive provision increase the quality of peer review? An experimental study

Abstract

Abstract. Although peer review is crucial for innovation and experimental discoveries in science, it is poorly understood in scientific terms. Discovering its true dynamics and exploring adjustments which improve the commitment of everyone involved could benefit scientific development for all disciplines and consequently increase innovation in the economy and the society. We have reported the results of an innovative experiment developed to model peer review. We demonstrate that offering material rewards to reviewers tends to decrease the quality and efficiency of the reviewing process. Our findings help to discuss the viability of different options of incentive provision, supporting the idea that journal editors and responsible of research funding agencies should be extremely careful in offering material incentives on reviewing, since these might undermine moral motives which guide reviewers' behavior.

Towards a Complexity-Friendly Policy: Breaking the Vicious Circle of Equilibrium Thinking in Economics and Public Policy

“…As argued by Bowles and Polanía-Reyes [11], policies enact interpretations of policy makers' intentions by decision makers and induce people to frame the decision environment. For example, incentivisation policies might induce decision makers to frame the context as a cost-benefit, rational choice problem, thus nurturing selfishness which may erode pre-existing good behaviour [46]. Furthermore, policies can compromise a control-adverse individual's sense of autonomy, dramatically affecting the process by which individuals learn new preferences and are socially digested through peer-to-peer social mechanisms (e.g.…”

The Miracle of Peer Review and Development in Science: An Agent-Based Model

“…The situation is of asymmetric information (Squazzoni et al 2013), in which the editor is unable to assess the true quality of submissions or the true effort of reviewers. The true quality of accepted papers is revealed probabilistically after publication (we fix the revelation probability P reveal ¼ 1 in the simulations reported) and there is no way for the editor to assess the true quality of rejected submissions.…”

“…Bianchi et al 2017;Squazzoni et al 2013). Given the presence of costs in terms of time and effort, no contribution (sloppy review) is the best reply strategy for reviews.…”

“…The importance of social incentives is highlighted by surveys asking for the motivations of reviewers (Malicki et al 2016;Warne 2016). In fact, monetary incentives might be in conflict with or drive out social incentives when applied to reviewers (Squazzoni et al 2013;Chetty et al 2014).…”

Free for Service: The Inadequate Incentives for Quality Peer Review

“…This is an interesting idea, but not a new one. Several journals, including JAMA, the NEJM and Annals of Internal Medicine have been doing it since 2004 [3], the Archives of Neurology since 2009 [12], and the European Respiratory Journal since 2010 [11]. Most offer credits only to reviews that meet certain quality criteria and are received on time.…”