Competition for novelty reduces information sampling in a research game - a registered report

Tiokhin, Leonid; Derex, Maxime

doi:10.1098/rsos.180934

Cited by 8 publications

(7 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another assumption-that scientists can only respond to competition by modifying their sample size and probability of abandonment-ignores other potential responses to competition (for example, increasing research effort 21,22,71 , but see refs. 23,72 ). Other extensions might allow questionable research practices 52 in response to competition.…”

Section: Discussionmentioning

confidence: 99%

“…In qualitative interviews, scientists admit to cutting corners in order to outcompete rivals 27 . In laboratory experiments using simple information-sampling paradigms, rewarding priority causes individuals to spend less time on exploration before making decisions between uncertain options 23,28 . More broadly, optimization models of scientists' behaviour suggest that, when novelty is disproportionately valued, scientists optimize their expected payoffs by conducting studies with low statistical power 6,29 .…”

mentioning

confidence: 99%

“…Although such research is suggestive, it has several limitations. By relying on general information-sampling paradigms in dyadic settings (for example, picking balls from urns 28 or revealing tiles to guess the majority colour on a grid 23 ), past experiments have missed critical features of scientific priority races, including the fact…”

mentioning

confidence: 99%

See 2 more Smart Citations

Competition for priority harms the reliability of science, but reforms can help

2021

Self Cite

View full text Add to dashboard Cite

cademic science is a culturally evolved social institution with formal rules, norms and conventions. However, in recent years, scientists have begun to examine the utility of even longstanding characteristics of this institution 1-3 . For example, it is now widely recognized that preferentially valuing positive over negative results can generate publication bias, which distorts the published literature 4,5 ; evaluating scientists based on their number of publications can cause a myopic focus on productivity at the expense of rigour 6 ; and rewarding scientists based on the prestige of the journal in which they publish may incentivize scientists to present their work in an overly positive light, submit low-quality papers to high-impact journals and engage in other questionable research practices [7][8][9][10][11][12] .The priority rule is a particularly longstanding scientific norm, in which individuals who are first to make discoveries receive disproportionate credit relative to all other individuals who provide solutions to the same problem 13,14 . Famously, Charles Darwin was motivated to publish his writings on evolution by means of natural selection in part because of a concern that he would lose priority to Alfred Russel Wallace, who had developed a similar idea. In his famous letter to Charles Lyell, Darwin proclaimed "I rather hate the idea of writing for priority, yet I certainly should be vexed if any one were to publish my doctrines before me" 15 . Rewards for priority take on various forms, including eponymy (that is, naming a scientific discovery after the scientist who discovered it), financial prizes (for example, the Nobel prize), an increased probability of publishing in high-impact journals, and better professional positions and speaking engagements 3,13,16,17 . Little research explicitly documents the career repercussions of losing a priority race (that is, getting scooped). However, one survey of physical and biological scientists found that over 60% of scientists reported being scooped at some point in their careers 18 , and a recent study among structural biologists found that scooped papers received 28% fewer citations and were 18% less likely to appear in a top-ten journal 19 . This suggests that scientists have significant incentives to compete over priority of discovery.Given its role as a major incentive, how does rewarding priority of discovery affect scientific inquiry? Rewards for priority can

show abstract

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Competition for priority harms the reliability of science, but reforms can help

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…workflow, hiring more individuals to work on a project, or allocating more of their working hours towards one problem as opposed to others). Empirical evidence suggests that competition increases effort in a variety of domains (21,22,79, but see 23,80). A useful extension could thus be to allow scientists to strategically adjust their research effort, in addition to their sample size and abandonment probability.…”

Section: Discussionmentioning

confidence: 99%

Competition for priority harms the reliability of science, but reforms can help

Tiokhin¹,

Mao²,

Morgan³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Incentives for priority of discovery are hypothesized to harm scientific reliability. Here, we evaluate this hypothesis by developing an evolutionary agent-based model of a competitive scientific process. We find that rewarding priority of discovery causes populations to culturally evolve towards conducting research with smaller samples. This reduces research reliability and the information-value of the average study. Increased startup costs for setting up single studies and increased payoffs for secondary results (a.k.a. “scoop protection”) attenuate the negative effects of competition. Further, large rewards for negative results promote the evolution of smaller sample sizes. Our results confirm the logical coherence of “scoop protection” reforms at several journals. Our results also imply that reforms to increase scientific efficiency, such as rapid journal turnaround times, may produce collateral damage by incentivizing lower-quality research; in contrast, reforms that increase startup costs, such as preregistration and registered reports, generate incentives for higher-quality research.

show abstract

“…Instead, this simply caused people to collect less information before offering their solution. Thus, the competition mainly caused them to rely on guesswork and reduced the accuracy of solutions offered (Tiokhin & Derex, 2019). Further, a study using an experimental game with multiple rounds found that cooperation was disrupted once performance rankings were introduced (Chambers & Baker, 2018).…”

Section: Science As a Sports Competitionmentioning

confidence: 99%

Science as collaborative knowledge generation

Ellemers

2020

British J Social Psychol

View full text Add to dashboard Cite

The COVID‐19 pandemic points to the need for scientists to pool their efforts in order to understand this disease and respond to the ensuing crisis. Other global challenges also require such scientific cooperation. Yet in academic institutions, reward structures and incentives are based on systems that primarily fuel the competition between (groups of) scientific researchers. Competition between individual researchers, research groups, research approaches, and scientific disciplines is seen as an important selection mechanism and driver of academic excellence. These expected benefits of competition have come to define the organizational culture in academia. There are clear indications that the overreliance on competitive models undermines cooperative exchanges that might lead to higher quality insights. This damages the well‐being and productivity of individual researchers and impedes efforts towards collaborative knowledge generation. Insights from social and organizational psychology on the side effects of relying on performance targets, prioritizing the achievement of success over the avoidance of failure, and emphasizing self‐interest and efficiency, clarify implicit mechanisms that may spoil valid attempts at transformation. The analysis presented here elucidates that a broader change in the academic culture is needed to truly benefit from current attempts to create more open and collaborative practices for cumulative knowledge generation.

show abstract

Competition for novelty reduces information sampling in a research game - a registered report

Cited by 8 publications

References 48 publications

Competition for priority harms the reliability of science, but reforms can help

Competition for priority harms the reliability of science, but reforms can help

Competition for priority harms the reliability of science, but reforms can help

Science as collaborative knowledge generation

Contact Info

Product

Resources

About