The effect of incentivization on the conjunction fallacy in judgments: A meta-analysis

Abstract

The conjunction fallacy is a classical judgment bias that was argued to be insusceptible to the positive effect of incentivization. We conducted a meta-analysis of the literature (n = 3,276) and found that although most studies did not report a significant effect of incentivization, the results across studies show a significant positive effect for incentivization, d = 0.19, with an odds ratio of 1.40 for answering correctly when incentivized. The effect was relatively smaller when examining absolute differences in the probability of correct judgment, implying that it is partly driven by studies with low baseline performance. Additionally, most of the variance between studies was estimated to be due to random noise. These findings join those of other judgment-bias studies to suggest a small but nevertheless robust debiasing effect of incentivization. The findings further suggest cautious interpretation of moderators and null results in light of the extreme variance between individual study results given a small effect size in the population.

Behavioral economics enhancers

“…Tversky and Kahneman (1983) argued that this bias is mainly driven by the representativeness heuristic, a fastand-frugal strategy for estimating probabilities based on similarity to representative examples (e.g., Linda seems more similar to a feminist bank teller than to any bank teller). Our meta-analysis of 11 conjunction fallacy studies (Yechiam and Zeif, 2023b) showed a small positive effect of incentivization on judgment performance in conjunction fallacy problems (d = 0.19 for all problems; d = 0.24 for the Linda problem). Again, disparities between different study results were mostly due to random noise.…”

“…In another meta-analysis (Yechiam and Zeif, 2023b), we examined the conjunction fallacy, one of the classical examples of judgment biases. Originally investigated by Tversky and Kahneman (1983), the conjunction fallacy is the tendency to estimate multiple contingencies occurring together as being more likely than one of the individual contingencies.…”

“…Nevertheless, an effect of baseline effort level may exist at the individual level. 9 As reviewed above, this has been scarcely examined, but some support is evident in the meta-analysis of base rate fallacy, which suggests a larger effect of incentives when baseline performance is poorer (Yechiam and Zeif, 2023b).…”