Choosing and learning: outcome valence differentially affects learning from free versus forced choices

Abstract

Positivity bias refers to learning more from positive than negative events. This learning asymmetry could either reflect a preference for positive events in general, or be the upshot of a more general, and perhaps, ubiquitous, "choice-confirmation" bias, whereby agents preferentially integrate information that confirms their previous decision. We systematically compared these two theories with 3 experiments mixing free-and forced-choice conditions, featuring factual and counterfactual learning and varying action requirements across "go" and "no-go" trials. Computational analyses of learning rates showed clear and robust evidence in favour of the "choice-confirmation" theory: participants amplified positive prediction errors in free-choice conditions while being valence-neutral on forced-choice conditions. We suggest that a choice-confirmation bias is adaptive to the extent that it reinforces actions that are most likely to meet an individual's needs, i.e. freely chosen actions. In contrast, outcomes from unchosen actions are more likely to be treated impartially, i.e. to be assigned no special value in self-determined decisions.

A normative account of confirmation bias during reinforcement learning

“…Humans have been observed to exhibit confirmatory biases when choosing between stimuli or actions that payout with uncertain probability (Chambon et al, 2019; Palminteri et al, 2017; Schuller et al, 2020). These biases drive participants to update positive outcomes (or those that are better than expected) for chosen options more sharply than negative outcomes, but to reverse this update pattern for the unchosen option.…”

“…In laboratory, confirmation bias has been studied with a variety of paradigms (Nickerson, 1998; Talluri, Urai, Tsetsos, Usher, & Donner, 2018). One paradigm in which the confirmation bias can be observed and measured involves reinforcement learning tasks, where participants have to learn from positive or negative feedback which options are worth taking (Chambon et al, 2019; Palminteri, Lefebvre, Kilford, & Blakemore, 2017), and this paper focusses on confirmation bias during reinforcement learning.…”

“…However, a different pattern of data was observed in follow-up studies in which counterfactual feedback was also offered – i.e., the participants were able to view the payout associated with both chosen and unchosen options. Following a feedback on the unchosen option, larger updates were observed for negative prediction errors (Chambon et al, 2019; Palminteri et al, 2017; Schuller et al, 2020). This is consistent with a confirmatory bias rather than a strictly optimistic bias, whereby belief revision helps to strengthen rather than weaken existing preconceptions about which option may be better.…”

The description-experience gap: a challenge for the neuroeconomics of decision-making under uncertainty

“…Risk-seeking attitudes have also been reported in the loss domain [91]. Risk-seeking behaviour in experience-based studies can be computationally explained by an increased sensitivity to positive (compared to negative) prediction errors ('positivity' bias) as the one generally documented in humans (Box 1) [67,92,93]. This hypothesis is corroborated by studies Table 1.…”

“…Conversely, the pessimistic bias implies risk-aversion. Both pessimistic and optimistic biases have been reported in the literature, with the latter bias being more frequently reported [67,92,93,114].…”