Rational metareasoning and the plasticity of cognitive control

Abstract

The human brain has the impressive capacity to adapt how it processes information to highlevel goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we develop and evaluate a model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert. We derive this model from a general theory according to which the function of cognitive control is to select and configure neural pathways so as to make optimal use of finite time and limited computational resources. The central idea of our Learned Value of Control model is that people use reinforcement learning to predict the value of candidate control signals of different types and intensities based on stimulus features. This model correctly predicts the learning and transfer effects underlying the adaptive control-demanding behavior observed in an experiment on visual attention and four experiments on interference control in Stroop and Flanker paradigms. Moreover, our model explained these findings significantly better than an associative learning model and a Win-Stay Lose-Shift model. Our findings elucidate how learning and experience might shape people's ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure. Author summaryThe human brain has the impressive ability to adapt how it processes information to high level goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we derive a computational model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert from a formal theory of the function of cognitive control. Across five experiments, we find that our model correctly predicts that people learn to adaptively regulate their attention and decision- Funding: This research was supported by the Office of Naval Research through Grant No. MURI N00014-13-1-0431 to TLG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests:The authors have declared that no competing interests exist.making and how these learning effects transfer to novel situations. Our findings elucidate how learning and experience might shape people's ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure.

A computational perspective on the roles of affect in cognitive control

“…As shown in Figure S1, the EVC model is able to reproduce 266 these classic observations, as well as the more basic observation that performance worsens 267 (slower and less accurate responding) on incongruent relative to congruent trials ( given performance-contingent reward (e.g., money or positive social feedback for completing a 280 task) than agents that anticipated lower rewards (Figure 2A). Consistent with analogous 281 simulations reported in previous work (Lieder et al, 2018;Musslick et al, 2015), we found that 282 increasing anticipatory affect predicts increased control allocation for equivalent rewards. As a 283 result, compared to agents that anticipated low rewards, agents that anticipated high rewards 284…”

“…directing information processing (Dreisbach & Fischer, 2012;Dreisbach & Fröber, 2018;Inzlicht 358 et al, 2015), normative theories of cognitive control have largely overlooked affect's role in 359 control allocation. In this study, we leveraged a computational implementation of the EVC theory 360 (Lieder et al, 2018;Musslick et al, 2015;Shenhav et al, 2013) to simulate several candidate 361 mechanisms through which cognitive control can be influenced by integral affect (e.g., 362…”

Motivation and Cognitive Control in Depression

“…One of the main advantages of the theory is its ability to integrate a wide range of behavioral and neural findings. The computational implementation of the theory has been shown to account for various effects associated with the allocation of cognitive control such as the sequential adaptation effects and post-error slowing (Musslick et al, 2015), including how individuals can learn about features of their environment that predict incentives for and demands of control allocation (Lieder et al, 2018).…”

“…After the original formulation of the EVC theory (Shenhav et al, 2013), the more recent work has developed a computational implementation of the theory (Lieder et al, 2018;Musslick et al, 2015). In the computational implementation, performance of each task (e.g., responding to the color or to the word of a Stroop stimulus) is implemented as a process of evidence accumulation toward a decision boundary.…”

“…The computational implementation of the EVC theory (Lieder et al, 2018;Musslick et al, 2015) has been successfully used to simulate a wide range of the existing empirical results in the domain of cognitive control (e.g. congruency effects, congruency sequence effects, taskswitching costs) and the influence of motivation on cognitive control (incentive-based enhancements in the processes such as inhibition, task-switching, and conflict adaptation).…”