Revisiting the effect of incentivization on cognitive reflection: A meta‐analysis

Yechiam, Eldad; Zeif, Dana

doi:10.1002/bdm.2286

Cited by 7 publications

(11 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additional moderators could not be examined because the differences between studies in effect size were driven mostly by random noise. This indicates that, first of all, there is a small effect of incentives that is about equal to that found in the recent meta-analysis of the Cognitive Reflection Test (Yechiam & Zeif, 2022). This is also consistent with the performance literature which typically reports a small to medium positive effect size of incentives on cognitive performance in lab studies (Cerasoli, Nicklin, & Ford, 2014;Shaw & Gupta, 2015).…”

Section: Discussionsupporting

confidence: 88%

“…Nevertheless, and despite the current results and those of Yechiam and Zeif (2022), one might argue that the effect of incentives on judgment biases is "slippery".…”

Section: Discussioncontrasting

confidence: 74%

“…Both of these meta-analyses showed a small positive effect of incentives (Cohen's d = 0.21), with some evidence that the effect is increased with the size of the potential payoff (Yechiam & Zeif, 2022). The present investigation extends this evaluation into 2 Reviews of this literature concluded either that incentives had a positive effect on judgment performance (Hertwig & Ortmann, 2001) or that it had a very limited effect (Camerer & Hogarth, 1999).…”

supporting

confidence: 53%

“…Though studies with low effect sizes have been criticized in psychology (see discussion in Cortina & Landis, 2009), it seems that some effects are robust despite being small, and while this can sometimes limit their applicability, this is not always the case. Especially, in the case of incentives, experimental studies typically use very small incentives (see e.g., Table 1 and Yechiam & Zeif, 2022), and though in the current study we could not examine the moderating role of incentive size due to the noise-driven heterogeneity between studies (and the low variance in the incentives provided), increasing the payoff size was previously found to boost the effect of incentivization on effort and performance (see e.g., Richter & Gendolla, 2009;Enke et al, 2022;Yechiam & Zeif, 2022).…”

Section: Discussionmentioning

confidence: 64%

“…In previous studies of cognitive performance, the size of outcomes was found to influence cognitive effort as evident by autonomic arousal and frontal cortical activation during the task (e.g., Wright, 1998;Richter & Gendolla, 2009;Xue et al, 2016), and the same could be the case for judgment biases. This is important in light of the fact that most studies of the effect of incentives on judgment biases typically use very small incentives (see review in Yechiam & Zeif, 2022).…”

mentioning

confidence: 99%

See 4 more Smart Citations

The effect of incentivization on the conjunction fallacy in judgments: A meta-analysis

Yechiam¹,

Zeif²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The conjunction fallacy is a classical judgment bias that was argued to be insusceptible to the positive effect of incentivization. We conducted a meta-analysis of the literature (n = 3,276) and found that although most studies did not report a significant effect of incentivization, the results across studies show a significant positive effect for incentivization, d = 0.19, with an odds ratio of 1.40 for answering correctly when incentivized. The effect was relatively smaller when examining absolute differences in the probability of correct judgment, implying that it is partly driven by studies with low baseline performance. Additionally, most of the variance between studies was estimated to be due to random noise. These findings join those of other judgment-bias studies to suggest a small but nevertheless robust debiasing effect of incentivization. The findings further suggest cautious interpretation of moderators and null results in light of the extreme variance between individual study results given a small effect size in the population.