Understanding Learning Through Uncertainty and Bias

Abstract

Learning allows humans and animals to make predictions about the environment and increases the likelihood of adaptive behavior. Studying the mechanisms through which humans regulate learning to update their predictions in the face of uncertainty can also shed light on maladaptive behaviors in various clinical and age groups. Drawing on normative learning models, we illustrate how learning should be calibrated to different sources of uncertainty, including perceptual uncertainty, risk, and environmental changes. We show that such models explain many hallmarks of human learning and highlight research on their underlying neural mechanisms. Systematic deviations from normative learning, which have been explored through computational psychiatry, provide a useful lens through which to understand psychiatric illness. In particular, normative models provide important insights into why different clinical and age groups show maladaptive learning behavior, thereby attempting to pinpoint the computational cause of dysfunctional decision making. We conclude by discussing relevant goals for future research.

Consistency is the key! Learning to adapt in a multi-context predictive inference task.

“…There has been considerable theoretical, experimental and computational development on this topic, resulting in a better understanding of the mechanisms underpinning optimal error driven learning and predictive inference (Brown & Steyvers, 2009;Bruckner, Heekeren, & Nassar, 2022;d'Acremont & Bossaerts, 2016;Marković & Kiebel, 2016;Mathys et al, 2014;Nassar, Wilson, Heasly, & Gold, 2010). Much of this work focuses on situations with some level of uncertainty.…”

“…However, if they are less sensitive, they may sometimes be pleased with temperatures substantially below or above their preferred temperature. These deviations are likely driven by your toddler's difficulty in discriminating the temperatures (perceptual uncertainty; Bruckner et al, 2022) and lead to variability in their exclamations, referred to as noise (Nassar et al, 2012(Nassar et al, , 2010 or risk (Bruckner et al, 2022). This variability contributes to your uncertainty about where in that range is the perfect temperature.…”

“…This variability contributes to your uncertainty about where in that range is the perfect temperature. This type of uncertainty is referred to as estimation uncertainty (Bruckner et al, 2022;Nassar et al, 2016;O'reilly, 2013;Payzan-LeNestour & Bossaerts, 2011). The more exclamations you get from your toddler, the better your estimate of their preferred temperature is.…”

“…The more exclamations you get from your toddler, the better your estimate of their preferred temperature is. In this way uncertainty reduces over time -this is referred to as relative uncertainty (Bruckner et al, 2022;Nassar et al, 2016).…”

“…Other similar models, like the Heirarchical Gaussian Filter (HGF; Mathys et al, 2014) also provide a measure of a learning rate in response to prediction errors under environments with uncertainty related to risk and volatility. For a comparison and description of the differences between these two main models see Marković and Kiebel (2016) and Bruckner et al (2022).…”