Does it Matter Which Effort Task You Use? A Comparison of Four Effort Tasks When Agents Compete for a Prize

Lezzi, Emanuela; Fleming, Piers; Zizzo, Daniel John

doi:10.2139/ssrn.2594659

Cited by 20 publications

(23 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another assumption-that scientists can only respond to competition by modifying their sample size and probability of abandonment-ignores other potential responses to competition (for example, increasing research effort 21,22,71 , but see refs. 23,72 ). Other extensions might allow questionable research practices 52 in response to competition.…”

Section: Discussionmentioning

confidence: 99%

Competition for priority harms the reliability of science, but reforms can help

2021

View full text Add to dashboard Cite

cademic science is a culturally evolved social institution with formal rules, norms and conventions. However, in recent years, scientists have begun to examine the utility of even longstanding characteristics of this institution 1-3 . For example, it is now widely recognized that preferentially valuing positive over negative results can generate publication bias, which distorts the published literature 4,5 ; evaluating scientists based on their number of publications can cause a myopic focus on productivity at the expense of rigour 6 ; and rewarding scientists based on the prestige of the journal in which they publish may incentivize scientists to present their work in an overly positive light, submit low-quality papers to high-impact journals and engage in other questionable research practices [7][8][9][10][11][12] .The priority rule is a particularly longstanding scientific norm, in which individuals who are first to make discoveries receive disproportionate credit relative to all other individuals who provide solutions to the same problem 13,14 . Famously, Charles Darwin was motivated to publish his writings on evolution by means of natural selection in part because of a concern that he would lose priority to Alfred Russel Wallace, who had developed a similar idea. In his famous letter to Charles Lyell, Darwin proclaimed "I rather hate the idea of writing for priority, yet I certainly should be vexed if any one were to publish my doctrines before me" 15 . Rewards for priority take on various forms, including eponymy (that is, naming a scientific discovery after the scientist who discovered it), financial prizes (for example, the Nobel prize), an increased probability of publishing in high-impact journals, and better professional positions and speaking engagements 3,13,16,17 . Little research explicitly documents the career repercussions of losing a priority race (that is, getting scooped). However, one survey of physical and biological scientists found that over 60% of scientists reported being scooped at some point in their careers 18 , and a recent study among structural biologists found that scooped papers received 28% fewer citations and were 18% less likely to appear in a top-ten journal 19 . This suggests that scientists have significant incentives to compete over priority of discovery.Given its role as a major incentive, how does rewarding priority of discovery affect scientific inquiry? Rewards for priority can

show abstract

Section: Discussionmentioning

confidence: 99%

Competition for priority harms the reliability of science, but reforms can help

2021

View full text Add to dashboard Cite

show abstract

“…workflow, hiring more individuals to work on a project, or allocating more of their working hours towards one problem as opposed to others). Empirical evidence suggests that competition increases effort in a variety of domains (21,22,79, but see 23,80). A useful extension could thus be to allow scientists to strategically adjust their research effort, in addition to their sample size and abandonment probability.…”

Section: Discussionmentioning

confidence: 99%

Competition for priority harms the reliability of science, but reforms can help

Tiokhin¹,

Mao²,

Morgan³

2020

Preprint

View full text Add to dashboard Cite

Incentives for priority of discovery are hypothesized to harm scientific reliability. Here, we evaluate this hypothesis by developing an evolutionary agent-based model of a competitive scientific process. We find that rewarding priority of discovery causes populations to culturally evolve towards conducting research with smaller samples. This reduces research reliability and the information-value of the average study. Increased startup costs for setting up single studies and increased payoffs for secondary results (a.k.a. “scoop protection”) attenuate the negative effects of competition. Further, large rewards for negative results promote the evolution of smaller sample sizes. Our results confirm the logical coherence of “scoop protection” reforms at several journals. Our results also imply that reforms to increase scientific efficiency, such as rapid journal turnaround times, may produce collateral damage by incentivizing lower-quality research; in contrast, reforms that increase startup costs, such as preregistration and registered reports, generate incentives for higher-quality research.

show abstract

“…Treatments in the Effort condition are identical to treatments in the No-Effort condition, except that participants need to solve a simple arithmetic problem before being able to click on a tile (figure 1 b ). This is a commonly used real-effort task in economics [35]. In this treatment, participants can increase their rate of clicking tiles by solving arithmetic problems more quickly (See Pilot study for checks on floor effects).…”

Section: Methodsmentioning

confidence: 99%

Competition for novelty reduces information sampling in a research game - a registered report

Tiokhin

Derex

2019

R. Soc. open sci.

View full text Add to dashboard Cite

Incentive structures shape scientists' research practices. One incentive in particular, rewarding priority of publication, is hypothesized to harm scientific reliability by promoting rushed, low-quality research. Here, we develop a laboratory experiment to test whether competition affects information sampling and guessing accuracy in a game that mirrors aspects of scientific investigation. In our experiment, individuals gather data in order to guess true states of the world and face a tradeoff between guessing quickly and increasing accuracy by acquiring more information. To test whether competition affects accuracy, we compare a treatment in which individuals are rewarded for each correct guess to a treatment where individuals face the possibility of being ‘scooped’ by a competitor. In a second set of conditions, we make information acquisition contingent on solving arithmetic problems to test whether competition increases individual effort (i.e. arithmetic-problem solving speed). We find that competition causes individuals to make guesses using less information, thereby reducing their accuracy ( H1a and H1b confirmed). We find no evidence that competition increases individual effort ( H2 , inconclusive evidence). Our experiment provides proof of concept that rewarding priority of publication can incentivize individuals to acquire less information, producing lower-quality research as a consequence.

show abstract

Does it Matter Which Effort Task You Use? A Comparison of Four Effort Tasks When Agents Compete for a Prize

Cited by 20 publications

References 59 publications

Competition for priority harms the reliability of science, but reforms can help

Competition for priority harms the reliability of science, but reforms can help

Competition for priority harms the reliability of science, but reforms can help

Competition for novelty reduces information sampling in a research game - a registered report

Contact Info

Product

Resources

About