Neural mechanisms of observational learning

Abstract

Individuals can learn by interacting with the environment and experiencing a difference between predicted and obtained outcomes (prediction error). However, many species also learn by observing the actions and outcomes of others. In contrast to individual learning, observational learning cannot be based on directly experienced outcome prediction errors. Accordingly, the behavioral and neural mechanisms of learning through observation remain elusive. Here we propose that human observational learning can be explained by two previously uncharacterized forms of prediction error, observational action prediction errors (the actual minus the predicted choice of others) and observational outcome prediction errors (the actual minus predicted outcome received by others). In a functional MRI experiment, we found that brain activity in the dorsolateral prefrontal cortex and the ventromedial prefrontal cortex respectively corresponded to these two distinct observational learning signals.prediction error | reward | vicarious learning | dorsolateral prefrontal cortex | ventromedial prefrontal cortex I n uncertain and changing environments, flexible control of actions has individual and evolutionary advantages by allowing goaldirected and adaptive behavior. Flexible action control requires an understanding of how actions bring about rewarding or punishing outcomes. Through instrumental conditioning, individuals can use previous outcomes to modify future actions (1-4). However, individuals learn not only from their own actions and outcomes but also from those that are observed. One of the most illustrative examples of observational learning happens in Antarctica, where flocks of Adelie penguins often congregate at the water's edge to enter the sea and feed on krill. However, the main predator of the penguins, the leopard seal, is often lurking out of sight beneath the waves, making it a risky prospect to be the first one to take the plunge. As this waiting game develops, one of the animals often becomes so hungry that it jumps, and if no seal appears the rest of the group will all follow suit. The following penguins make a decision after observing the action and outcome of the first (5). This ability to learn from observed actions and outcomes is a pervasive feature of many species and can be absolutely crucial when the stakes are high. For example, predator avoidance techniques or the eating of a novel food item are better learned from another's experience rather than putting oneself at risk with trial-and-error learning. Although we know a fair amount about the neural mechanisms of individuals learning about their own actions and outcomes (6), almost nothing is known about the brain processes involved when individuals learn from observed actions and outcomes (7). This lack of knowledge is all the more surprising given that observational learning is such a wide-ranging phenomenon.In this study, 21 participants engaged in a novel observational learning task based on a simple two-armed bandit problem (Fig. 1A) while being scanne...

Neurocomputational mechanisms of prosocial learning and links to empathy

“…Learning behavior in the self, prosocial, and no one conditions was modeled using a reinforcement learning (RL) algorithm (2), which has been extensively used to examine the behavioral and neural basis of arbitrary visuomotor associations in both self and social contexts (13,(17)(18)(19). The RL model assumes that the associative value of an action (or stimulus) changes when new information reveals that the actual outcome of a decision is different from the expected outcome (2).…”

“…The ventral striatum, sgACC, dACC, and DLPFC are also implicated in processing information about rewards others will receive (12-16), PEs when interacting with others (13,(17)(18)(19), and prosocial behavior (e.g., refs. 4-7 for reviews).…”

The role of learning-related dopamine signals in addiction vulnerability

“…In humans, functional MRI (fMRI) studies and even in vivo surgical recordings (Zaghloul et al, 2009) have reported correlates of prediction-error signals in the striatum that resemble those of dopamine neurons recorded in animals, including phasic (event-related) positive and negative prediction error responses (McClure et al, 2003a;O'Doherty et al, 2003;D'Ardenne et al, 2008) that reflect probability (e.g. Abler et al 2006;Burke et al 2010;Spicer et al 2007;Tobler et al 2007) and more specific predictions of formal learning theories Kahnt et al, 2012;Rutledge et al, 2010;Tobler et al, 2007). However, it is worth keeping in mind that the haemodynamic response measured with neuroimaging is nonspecific rather than a one-to-one reflection of a particular neural event such as dopamine release (see also Düzel et al 2009), which could explain why some fMRI studies have suggested positive coding of losses (Seymour et al 2004; although see also Tom et al 2007) and a dominance of action over value (Guitart-Masip et al, 2012 (Kamin, 1969) used an aversive between-subjects design; by contrast, the experiment described in the text and depicted in abbreviated form here (Waelti et al, 2001) used an appetitive within-subject design where the test consists of a comparison between Y and X (see also Figure 2A); The optogenetic unblocking experiment of Steinberg et al (2013) used a between-subject design.…”

Spatial gradient in value representation along the medial prefrontal cortex reflects individual differences in prosociality

“…Moreover, despite the rapidly accumulating research on reward processing in social domains (13)(14)(15)(16)(17)(18)(19), the question remains of how neural representation of self-regarding vs. other-regarding values is related to individual differences in altruistic behavior.…”