Competition for priority harms the reliability of science, but reforms can help

Abstract

Incentives for priority of discovery are hypothesized to harm scientific reliability. Here, we evaluate this hypothesis by developing an evolutionary agent-based model of a competitive scientific process. We find that rewarding priority of discovery causes populations to culturally evolve towards conducting research with smaller samples. This reduces research reliability and the information-value of the average study. Increased startup costs for setting up single studies and increased payoffs for secondary results (a.k.a. “scoop protection”) attenuate the negative effects of competition. Further, large rewards for negative results promote the evolution of smaller sample sizes. Our results confirm the logical coherence of “scoop protection” reforms at several journals. Our results also imply that reforms to increase scientific efficiency, such as rapid journal turnaround times, may produce collateral damage by incentivizing lower-quality research; in contrast, reforms that increase startup costs, such as preregistration and registered reports, generate incentives for higher-quality research.

The past, present and future of Registered Reports

“…Perhaps the greatest limitation of the RR format is the time taken for submissions to be reviewed at stage 1 and receive IPA, thus delaying the commencement of research (see the section "Limitations and drawbacks"). While it can be argued that elimination of publication bias offsets this cost at a community level, and that the stage 1 delay could improve quality by increasing start-up costs 81 , this downtime nevertheless reduces accessibility of the format to individual researchers and can make it prohibitive for short-term projects. Here, we consider four innovations that could substantially accelerate stage 1 review without reducing quality.…”

Interdisciplinarity Can Aid the Spread of Better Methods Between Scientific Communities

“…Or this could stem from the differential success of students whose advisors use credit-producing methods. Previous models have shown that these sorts of selection processes can help explain failures of methodology and discovery in science (Smaldino and McElreath, 2016;O'Connor, 2019;Holman and Bruner, 2017;Stewart and Plotkin, 2020;Tiokhin et al, 2020).…”