Statistical learning and adaptive decision-making underlie human response time variability in inhibitory control

Ma, Ning; Yu, Angela J.

doi:10.3389/fpsyg.2015.01046

Cited by 11 publications

(21 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both HSC and MDI behave as though they believe the environment to be changeable—in fact, at approximately once every 1/(1-

) = 1/(1-0.6) = 2.5 trials—instead of assuming the reward rates to be static, which was the “true” experimental design. The estimated ∩γ is smaller than values found in most other tasks, which tends to be around 0.7–0.8 (Yu and Cohen, 2009 ; Ide et al, 2013 ; Yu and Huang, 2014 ; Zhang et al, 2014 ; Ma and Yu, 2015 ), and may potentially be due to the longer inter-trial interval used in this task (temporal discounting may be influenced also by absolute time, not only discrete trials as assumed in DBM).…”

Section: Resultsmentioning

confidence: 72%

Altered Statistical Learning and Decision-Making in Methamphetamine Dependence: Evidence from a Two-Armed Bandit Task

et al. 2015

Self Cite

View full text Add to dashboard Cite

Understanding how humans weigh long-term and short-term goals is important for both basic cognitive science and clinical neuroscience, as substance users need to balance the appeal of an immediate high vs. the long-term goal of sobriety. We use a computational model to identify learning and decision-making abnormalities in methamphetamine-dependent individuals (MDI, n = 16) vs. healthy control subjects (HCS, n = 16), in a two-armed bandit task. In this task, subjects repeatedly choose between two arms with fixed but unknown reward rates. Each choice not only yields potential immediate reward but also information useful for long-term reward accumulation, thus pitting exploration against exploitation. We formalize the task as comprising a learning component, the updating of estimated reward rates based on ongoing observations, and a decision-making component, the choice among options based on current beliefs and uncertainties about reward rates. We model the learning component as iterative Bayesian inference (the Dynamic Belief Model), and the decision component using five competing decision policies: Win-stay/Lose-shift (WSLS), ε-Greedy, τ-Switch, Softmax, Knowledge Gradient. HCS and MDI significantly differ in how they learn about reward rates and use them to make decisions. HCS learn from past observations but weigh recent data more, and their decision policy is best fit as Softmax. MDI are more likely to follow the simple learning-independent policy of WSLS, and among MDI best fit by Softmax, they have more pessimistic prior beliefs about reward rates and are less likely to choose the option estimated to be most rewarding. Neurally, MDI's tendency to avoid the most rewarding option is associated with a lower gray matter volume of the thalamic dorsal lateral nucleus. More broadly, our work illustrates the ability of our computational framework to help reveal subtle learning and decision-making abnormalities in substance use.

show abstract

“…Both HSC and MDI behave as though they believe the environment to be changeable—in fact, at approximately once every 1/(1-

Section: Resultsmentioning

confidence: 72%

Altered Statistical Learning and Decision-Making in Methamphetamine Dependence: Evidence from a Two-Armed Bandit Task

et al. 2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…In a second experiment, 8- and 9-year-olds' math achievement was not uniquely predicted by magnitude comparison accuracy, over and above inhibitory control. The authors therefore argued that the relationship between the ability to compare magnitudes and math achievement could be accounted for by individual differences in inhibitory control (but see Keller and Libertus, 2015 , for contradictory findings).…”

Section: Introductionmentioning

confidence: 99%

Of Huge Mice and Tiny Elephants: Exploring the Relationship Between Inhibitory Processes and Preschool Math Skills

2016

View full text Add to dashboard Cite

The cognitive mechanisms underpinning the well-established relationship between inhibitory control and early maths skills remain unclear. We hypothesized that a specific aspect of inhibitory control drives its association with distinct math skills in very young children: the ability to ignore stimulus dimensions that are in conflict with task-relevant representations. We used an Animal Size Stroop task in which 3- to 6-year-olds were required to ignore the physical size of animal pictures to compare their real-life dimensions. In Experiment 1 (N = 58), performance on this task correlated with standardized early mathematics achievement. In Experiment 2 (N = 48), performance on the Animal Size Stroop task related to the accuracy of magnitude comparison, specifically for trials on which the physical size of dot arrays was incongruent with their numerosity. This highlights a process-oriented relationship between interference control and resolving conflict between discrete and continuous quantity, and in turn calls for further detailed empirical investigations of whether, how and why inhibitory processes matter to emerging numerical cognition.

show abstract

“…by discretizing belief state space [19] or averaging over possible change point times [1]) is both difficult and (thus) unnecessary for making Bayes-optimal predictions. In practice, implementing the m-ary DBM by discretization of the belief state-space is common practice [19,22,8,7], which has a computational and representational complexity of O(e km ) per observation, where k depends on fineness of the discretization, while the near exact-Bayes approximation EXP is only O(m).…”

Section: Discussionmentioning

confidence: 99%

“…While the analytical relationship we derive between DBM and EXP breaks down when change points are rare (α ≈ 1), this seems to be irrelevant in tasks where human behavior has been shown to be fitted well by DBM, since fitted α ranges between 0.7 and 0.8 [19,22,23,8,7]). Finally, we note that our approximation technique does not preclude an approximation in which the learning rate is modulated from trial to trial.…”

Section: Discussionmentioning

confidence: 99%

Demystifying excessively volatile human learning: A Bayesian persistent prior and a neural approximation

Ryali

Reddy

2016

Preprint

Self Cite

View full text Add to dashboard Cite

Understanding how humans and animals learn about statistical regularities in stable and volatile environments, and utilize these regularities to make predictions and decisions, is an important problem in neuroscience and psychology. Using a Bayesian modeling framework, specifically the Dynamic Belief Model (DBM), it has previously been shown that humans tend to make the default assumption that environmental statistics undergo abrupt, unsignaled changes, even when environmental statistics are actually stable. Because exact Bayesian inference in this setting, an example of switching state space models, is computationally intensive, a number of approximately Bayesian and heuristic algorithms have been proposed to account for learning/prediction in the brain. Here, we examine a neurally plausible algorithm, a special case of leaky integration dynamics we denote as EXP (for exponential filtering), that is significantly simpler than all previously suggested algorithms except for the delta-learning rule, and which far outperforms the delta rule in approximating Bayesian prediction performance. We derive the theoretical relationship between DBM and EXP, and show that EXP gains computational efficiency by foregoing the representation of inferential uncertainty (as does the delta rule), but that it nevertheless achieves near-Bayesian performance due to its ability to incorporate a "persistent prior" influence unique to DBM and absent from the other algorithms. Furthermore, we show that EXP is comparable to DBM but better than all other models in reproducing human behavior in a visual search task, suggesting that human learning and prediction also incorporates an element of persistent prior. More broadly, our work demonstrates that when observations are information-poor, detecting changes or modulating the learning rate is both difficult and (thus) unnecessary for making Bayes-optimal predictions.

show abstract

Statistical learning and adaptive decision-making underlie human response time variability in inhibitory control

Cited by 11 publications

References 23 publications

Altered Statistical Learning and Decision-Making in Methamphetamine Dependence: Evidence from a Two-Armed Bandit Task

Altered Statistical Learning and Decision-Making in Methamphetamine Dependence: Evidence from a Two-Armed Bandit Task

Of Huge Mice and Tiny Elephants: Exploring the Relationship Between Inhibitory Processes and Preschool Math Skills

Demystifying excessively volatile human learning: A Bayesian persistent prior and a neural approximation

Contact Info

Product

Resources

About