Strategy selection as rational metareasoning

Lieder, Falk; Griffiths, Tom

doi:10.31219/osf.io/6yugk

Cited by 23 publications

(31 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the LVOC model's sophistication may be necessary to explain more complex phenomena such as how people learn to orchestrate their thoughts to solve complex problems and acquire sophisticated cognitive strategies. Recent work has indeed shown that the learning mechanism instantiated by the LVOC model can also capture aspects of how people learn how to plan [70] and to flexibly and adaptively choose between alternative cognitive strategies [53]. Testing whether people learn to select sequences of control signals in the way predicted by our model is an interesting direction for future research.…”

Section: Discussionmentioning

confidence: 95%

“…In many real-world scenarios and some experiments, the opportunity cost is time-varying. This can be incorporated into our model by adding a learning mechanism that estimates ω from experience [52,53]. Following [46], we model the intrinsic cost of control as the implementation cost…”

Section: Krebs Et Al (2010) Expmentioning

confidence: 99%

“…The model developed in this article builds on two previous theories: the EVC theory, which offered a normative account of control specification [7], and the rational metareasoning theory of strategy selection [53], which suggested that people acquire the capacity to select heuristics adaptively by learning a predictive model of the execution time and accuracy of those heuristics. The LVOC model synergistically integrates these two theories: it augments the EVC theory with the metacognitive learning and prediction mechanisms identified by [53], and it augments rational metareasoning models of strategy selection with the capacity to specify continuous control signals that gradually adjust parameters of the controlled process (see S2 Text).…”

Section: The Lvoc Model Integrates Control Specification and Strategymentioning

confidence: 99%

“…The LVOC model synergistically integrates these two theories: it augments the EVC theory with the metacognitive learning and prediction mechanisms identified by [53], and it augments rational metareasoning models of strategy selection with the capacity to specify continuous control signals that gradually adjust parameters of the controlled process (see S2 Text).…”

Section: The Lvoc Model Integrates Control Specification and Strategymentioning

confidence: 99%

See 3 more Smart Citations

Rational metareasoning and the plasticity of cognitive control

Lieder¹,

Shenhav²,

Musslick³

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

The human brain has the impressive capacity to adapt how it processes information to highlevel goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we develop and evaluate a model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert. We derive this model from a general theory according to which the function of cognitive control is to select and configure neural pathways so as to make optimal use of finite time and limited computational resources. The central idea of our Learned Value of Control model is that people use reinforcement learning to predict the value of candidate control signals of different types and intensities based on stimulus features. This model correctly predicts the learning and transfer effects underlying the adaptive control-demanding behavior observed in an experiment on visual attention and four experiments on interference control in Stroop and Flanker paradigms. Moreover, our model explained these findings significantly better than an associative learning model and a Win-Stay Lose-Shift model. Our findings elucidate how learning and experience might shape people's ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure. Author summaryThe human brain has the impressive ability to adapt how it processes information to high level goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we derive a computational model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert from a formal theory of the function of cognitive control. Across five experiments, we find that our model correctly predicts that people learn to adaptively regulate their attention and decision- Funding: This research was supported by the Office of Naval Research through Grant No. MURI N00014-13-1-0431 to TLG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests:The authors have declared that no competing interests exist.making and how these learning effects transfer to novel situations. Our findings elucidate how learning and experience might shape people's ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure.

show abstract

Section: Discussionmentioning

confidence: 95%

Section: Krebs Et Al (2010) Expmentioning

confidence: 99%

Section: The Lvoc Model Integrates Control Specification and Strategymentioning

confidence: 99%

Section: The Lvoc Model Integrates Control Specification and Strategymentioning

confidence: 99%

See 2 more Smart Citations

Rational metareasoning and the plasticity of cognitive control

Lieder¹,

Shenhav²,

Musslick³

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Choosing a cognitive system can be construed as metareasoning: deciding how to deploy cognitive resources in an efficient manner, considering both the expected utility and expected costs rather than using the most accurate strategy regardless of resource usage (Russell and Wefald 1991). This framework has been shown to explain general human strategy selection (Lieder and Griffiths 2017;Payne et al 1988) and has been used to describe how people allocate cognitive resources within a strategy for physical prediction (Hamrick et al 2015).…”

Section: Metareasoning Over Cognitive Systemsmentioning

confidence: 99%

Different Physical Intuitions Exist Between Tasks, Not Domains

2018

View full text Add to dashboard Cite

Does human behavior exploit deep and accurate knowledge about how the world works, or does it rely on shallow and often inaccurate heuristics? This fundamental question is rooted in a classic dichotomy in psychology: human intuitions about even simple scenarios can be poor, yet their behaviors can exceed the capabilities of even the most advanced machines. One domain where such a dichotomy has classically been demonstrated is intuitive physics. Here we demonstrate that this dichotomy is rooted in how physical knowledge is measured: extrapolation of ballistic motion is idiosyncratic and erroneous when people draw the trajectories but consistent with accurate physical inferences under uncertainty when people use the same trajectories to catch a ball or release it to hit a target. Our results suggest that the contrast between rich and calibrated versus poor and inaccurate patterns of physical reasoning exists as a result of using different systems of knowledge across tasks, rather than being driven solely by a universal system of knowledge that is inconsistent across physical principles.

show abstract