Information about action outcomes differentially affects learning from self-determined versus imposed choices

Chambon, Valérian; Théro, Héloïse; Vidal, Marie; Vandendriessche, Henri; Haggard, Patrick; Palminteri, Stefano

doi:10.1038/s41562-020-0919-5

Cited by 71 publications

(157 citation statements)

References 42 publications

Supporting

Mentioning

136

Contrasting

Order By: Relevance

“…Risk-seeking behaviour in experience-based studies can be computationally explained by an increased sensitivity to positive (compared to negative) prediction errors ('positivity' bias) which is generally documented in human reinforcement learning (box 1) [72][73][74]. This hypothesis is corroborated by studies demonstrating a stronger impact of past positive outcome in choices using either model-free or model-based measures [81,82,101].…”

Section: Decision Under Risk In Monkeysmentioning

confidence: 90%

The description–experience gap: a challenge for the neuroeconomics of decision-making under uncertainty

Garcia

Cerrotti

Palminteri

2021

Phil. Trans. R. Soc. B

Self Cite

View full text Add to dashboard Cite

The experimental investigation of decision-making in humans relies on two distinct types of paradigms, involving either description- or experience-based choices. In description-based paradigms, decision variables (i.e. payoffs and probabilities) are explicitly communicated by means of symbols. In experience-based paradigms decision variables are learnt from trial-by-trial feedback. In the decision-making literature, ‘description–experience gap’ refers to the fact that different biases are observed in the two experimental paradigms. Remarkably, well-documented biases of description-based choices, such as under-weighting of rare events and loss aversion, do not apply to experience-based decisions. Here, we argue that the description–experience gap represents a major challenge, not only to current decision theories, but also to the neuroeconomics research framework, which relies heavily on the translation of neurophysiological findings between human and non-human primate research. In fact, most non-human primate neurophysiological research relies on behavioural designs that share features of both description- and experience-based choices. As a consequence, it is unclear whether the neural mechanisms built from non-human primate electrophysiology should be linked to description-based or experience-based decision-making processes. The picture is further complicated by additional methodological gaps between human and non-human primate neuroscience research. After analysing these methodological challenges, we conclude proposing new lines of research to address them. This article is part of the theme issue ‘Existence and prevalence of economic behaviours among non-human primates’.

show abstract

Section: Decision Under Risk In Monkeysmentioning

confidence: 90%

The description–experience gap: a challenge for the neuroeconomics of decision-making under uncertainty

Garcia

Cerrotti

Palminteri

2021

Phil. Trans. R. Soc. B

Self Cite

View full text Add to dashboard Cite

show abstract

“…Past work has provided critical insights into related aspects of how people adapt control based on learned incentives. For instance, a substantial body of research has examined how people learn about the amount and frequency of reward in their environment (Daw & Tobler, 2013;Dolan & Dayan, 2013;Niv et al, 2006), including the extent to which learning differs depending on the valence of feedback (e.g., whether an outcome was better or worse than expected; Chambon et al, 2020;Collins & Frank, 2014;Garrett & Daw, 2020;Lefebvre et al, Learning when effort matters 4 2017). Recent work has shown that people not only adapt their actions to these changing rewards, but also their control states (Lieder et al, 2018).…”

mentioning

confidence: 99%

Learning when effort matters: Neural dynamics underlying updating and adaptation to changes in performance efficacy

Grahek

Frömer

Shenhav

2020

Preprint

View full text Add to dashboard Cite

To determine how much cognitive control to invest in a task, people need to consider whether exerting control matters for obtaining potential rewards. In particular, they need to account for the efficacy of their performance – the degree to which potential rewards are determined by their performance or by independent factors (e.g., random chance). Yet it remains unclear how people learn about their performance efficacy in a given environment. Here, we examined the neural and computational mechanisms through which people (a) learn and dynamically update efficacy expectations in a changing environment, and (b) proactively adjust control allocation based on their current efficacy expectations. We recorded EEG in 40 participants performing an incentivized cognitive control task, while their performance efficacy (the likelihood that reward for a given trial would be determined by performance or at random) dynamically varied over time. We found that participants continuously updated their self- reported efficacy expectations based on recent feedback, and that these updates were well described by a standard prediction error-based reinforcement learning algorithm. Paralleling findings on updating of expected rewards, we found that model-based estimates of efficacy prediction errors were encoded by the feedback-related P3b. Updated expectations of efficacy in turn influenced levels of effort exerted on subsequent trials, reflected in greater proactive control (indexed by the contingent negative variation [CNV]) and improved performance when participants expected their performance to be more efficacious. These findings demonstrate that learning and adaptation to the efficacy of one’s environment is underpinned by similar computations and neural mechanisms as are involved in learning about potential reward.

show abstract

“…This precise manipulation of information presumably explains why we did not observe substantial differences in overall decision accuracy between cue-based and outcome-based conditions. Moreover, the concept of controllable information sampling does not map onto a classical distinction between free and forced choices (Chambon et al, 2020;Sidarus et al, 2019;Wilson et al, 2014). Forced-choice experiments are typically conducted in otherwise controllable environments, and the sequences resulting from free and forced choices therefore vastly differ in terms of the information available for subsequent choices.…”

Section: Discussionmentioning

confidence: 99%

“…In 'passive sampling' paradigms, participants are observers who sample information over which they have no instrumental control (Murphy et al, 2016;van den Berg et al, 2016;Zylberberg et al, 2018), which is typically the case in perceptual decision-making studies (for reviews, see (Gold and Shadlen, 2007;Hanks and Summerfield, 2017)). In contrast, 'active sampling' paradigms allow participants to choose intentionally which information source to sample (Chambon et al, 2020;Gureckis and Markant, 2012;Markant and Gureckis, 2014;Rouault et al, 2019b). Unlike passive sampling, active sampling provide participants with instrumental control over the information used to form and revise beliefs.…”

mentioning

confidence: 99%

Controllability boosts neural and cognitive signatures of changes-of-mind in uncertain environments

Rouault

Weiss

Lee

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In uncertain environments, seeking information about the accuracy of alternative strategies is essential for adapting behavior to changes in task contingencies. However, information seeking often co-occurs with changes-of-mind about the perceived accuracy of the current strategy, making it difficult to isolate its specific mechanisms. Here we leveraged the fact that genuine information seeking requires instrumental control to study its cognitive signatures in an adaptive decision-making task tested with and without control. We found that changes-of-mind occurring in controllable environments require more evidence against the current strategy, are associated with reduced confidence, but are nevertheless more likely to be confirmed on the next decision. Computational modelling explained these effects of information seeking through a decrease in the perceived volatility of controllable environments, resulting in stronger and more prolonged effects of changes-of-mind on cognition and behavior. Together, these findings explain the high degree of subjective uncertainty associated with information seeking.

show abstract

Information about action outcomes differentially affects learning from self-determined versus imposed choices

Cited by 71 publications

References 42 publications

The description–experience gap: a challenge for the neuroeconomics of decision-making under uncertainty

The description–experience gap: a challenge for the neuroeconomics of decision-making under uncertainty

Learning when effort matters: Neural dynamics underlying updating and adaptation to changes in performance efficacy

Controllability boosts neural and cognitive signatures of changes-of-mind in uncertain environments

Contact Info

Product

Resources

About