Cognitive Control as Cost‐Benefit Decision Making

Kool, Wouter; Shenhav, Amitai; Botvinick, Matthew

doi:10.1002/9781118920497.ch10

Cited by 66 publications

(65 citation statements)

References 154 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Building on previous work modeling the specification of cognitive control in terms of metadecision making [12][13][14]29,33,65,66] and reinforcement learning [33,34,[67][68][69], we have illustrated that at least some of the functions of cognitive control can be characterized using the formal framework of rational metareasoning [26] and meta-level Markov decision processes [27]. Concretely, modeling the function of cognitive control as a meta-level MDP allowed us to derive the first formal computational model of how people learn to specify continuous control signals and how these learning effects transfer to novel situations.…”

Section: Discussionmentioning

confidence: 99%

“…In broad accord with the predictions of the EVC theory, previous research has found that control specification is context-sensitive [8,9] and modulated by reward across multiple domains [10,11], such as attention, response inhibition, interference control, and task switching. While previous theories account for that fact that people's performance in these task is sensitive to reward [7,[12][13][14], it remains unclear how these dependencies arise from people's experience. Recently, it has been proposed that the underlying mechanism is associative learning [15,16].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Rational metareasoning and the plasticity of cognitive control

Lieder¹,

Shenhav²,

Musslick³

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

The human brain has the impressive capacity to adapt how it processes information to highlevel goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we develop and evaluate a model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert. We derive this model from a general theory according to which the function of cognitive control is to select and configure neural pathways so as to make optimal use of finite time and limited computational resources. The central idea of our Learned Value of Control model is that people use reinforcement learning to predict the value of candidate control signals of different types and intensities based on stimulus features. This model correctly predicts the learning and transfer effects underlying the adaptive control-demanding behavior observed in an experiment on visual attention and four experiments on interference control in Stroop and Flanker paradigms. Moreover, our model explained these findings significantly better than an associative learning model and a Win-Stay Lose-Shift model. Our findings elucidate how learning and experience might shape people's ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure. Author summaryThe human brain has the impressive ability to adapt how it processes information to high level goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we derive a computational model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert from a formal theory of the function of cognitive control. Across five experiments, we find that our model correctly predicts that people learn to adaptively regulate their attention and decision- Funding: This research was supported by the Office of Naval Research through Grant No. MURI N00014-13-1-0431 to TLG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests:The authors have declared that no competing interests exist.making and how these learning effects transfer to novel situations. Our findings elucidate how learning and experience might shape people's ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Rational metareasoning and the plasticity of cognitive control

Lieder¹,

Shenhav²,

Musslick³

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The allocation of effort can be operationalized as a value-based decision, with the type and amount of effort determined by weighing attendant costs and benefits [19,20,129]. For instance, early psychological models of motivation proposed that the decision to engage in an effortful task should be a function of the incentives for a task and one’s perception of their likelihood of achieving the goal in question [18,130].…”

Section: Figure Imentioning

confidence: 99%

The Effort Paradox: Effort Is Both Costly and Valued

Inzlicht

Shenhav

Olivola

2018

Trends in Cognitive Sciences

Self Cite

553

353

View full text Add to dashboard Cite

According to prominent models in cognitive psychology, neuroscience, and economics, effort (be it physical or mental) is costly: when given a choice, humans and non-human animals alike tend to avoid effort. Here, we suggest that the opposite is also true and review extensive evidence that effort can also add value. Not only can the same outcomes be more rewarding if we apply more (not less) effort, sometimes we select options precisely because they require effort. Given the increasing recognition of effort’s role in motivation, cognitive control, and value-based decision-making, considering this neglected side of effort will not only improve formal computational models, but also provide clues about how to promote sustained mental effort across time.

show abstract

“…Although such proactive control can facilitate the processing of upcoming events, it is thought to be metabolically costly and is therefore minimized in nonmotivated conditions (Braver, 2012). In motivated conditions, the presumed costs of control may be offset by increased incentives (Kool, Shenhav, & Botvinick, 2017). Consequently, proactive control may be applied in a more continuous manner (Esterman et al, 2016;Jimura et al, 2010;Locke & Braver, 2008).…”

Section: Reward Motivation Reduces Implicit Temporal Bias In Reactimentioning

confidence: 99%

Motivation alters implicit temporal attention through sustained and transient mechanisms: A behavioral and pupillometric study

et al. 2018

View full text Add to dashboard Cite

Temporal expectations aid performance by allowing the optimization of attentional readiness at moment of highest target probability. Reward enhances cognitive performance through its action on preparatory and reactive attentional processes. To elucidate how motivation interacts with mechanisms of implicit temporal attention, we studied healthy young adult participants (N = 73) performing a sustained attention task with simultaneous pupillometric recording, under different reward conditions (baseline: 0 c; reward: 10 c/fast response). Target timing was temporally unpredictable (variable foreperiod: 2-10 s, uniformly distributed), in which case implicitly formed timing expectations. Trials were binned according to current foreperiod (FP ; short: 2-6 s; long: 6-10 s) and preceding foreperiod (FP ; short: 2-6 s; long: 6-10 s). Overall, performance data showed the expected temporal attention effects, with slower responses after shorter FP s, particularly when they followed longer FP s. Moreover, these temporal effects were significantly reduced in the reward condition. While performance improved in all trial types, the largest benefit appeared in trials that were normally most disadvantaged by invalid temporal expectation. Furthermore, reward motivation was accompanied by an increase in sustained (prestimulus) and transient (poststimulus response) pupil diameter. The latter effect was particularly evident following short FP s. The current findings suggest that reward motivation can improve overall attentional performance and reduce implicit temporal bias, both through preparatory and reactive attentional mechanisms.

show abstract

Cognitive Control as Cost‐Benefit Decision Making

Cited by 66 publications

References 154 publications

Rational metareasoning and the plasticity of cognitive control

Rational metareasoning and the plasticity of cognitive control

The Effort Paradox: Effort Is Both Costly and Valued

Motivation alters implicit temporal attention through sustained and transient mechanisms: A behavioral and pupillometric study

Contact Info

Product

Resources

About