Learning to Simulate Others' Decisions

Suzuki, Shinsuke; Harasawa, Norihiro; Ueno, Kenichi; Gardner, Justin L.; Ichinohe, Noritaka; Haruno, Masahiko; Cheng, Kang; Nakahara, Hiroyuki

doi:10.1016/j.neuron.2012.04.030

Cited by 204 publications

(233 citation statements)

References 58 publications

Supporting

Mentioning

206

Contrasting

Unclassified

Order By: Relevance

“…Learning behavior in the self, prosocial, and no one conditions was modeled using a reinforcement learning (RL) algorithm (2), which has been extensively used to examine the behavioral and neural basis of arbitrary visuomotor associations in both self and social contexts (13,(17)(18)(19). The RL model assumes that the associative value of an action (or stimulus) changes when new information reveals that the actual outcome of a decision is different from the expected outcome (2).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Neurocomputational mechanisms of prosocial learning and links to empathy

Lockwood

Apps

Valton

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

186

236

View full text Add to dashboard Cite

Reinforcement learning theory powerfully characterizes how we learn to benefit ourselves. In this theory, prediction errors-the difference between a predicted and actual outcome of a choice-drive learning. However, we do not operate in a social vacuum. To behave prosocially we must learn the consequences of our actions for other people. Empathy, the ability to vicariously experience and understand the affect of others, is hypothesized to be a critical facilitator of prosocial behaviors, but the link between empathy and prosocial behavior is still unclear. During functional magnetic resonance imaging (fMRI) participants chose between different stimuli that were probabilistically associated with rewards for themselves (self), another person (prosocial), or no one (control). Using computational modeling, we show that people can learn to obtain rewards for others but do so more slowly than when learning to obtain rewards for themselves. fMRI revealed that activity in a posterior portion of the subgenual anterior cingulate cortex/basal forebrain (sgACC) drives learning only when we are acting in a prosocial context and signals a prosocial prediction error conforming to classical principles of reinforcement learning theory. However, there is also substantial variability in the neural and behavioral efficiency of prosocial learning, which is predicted by trait empathy. More empathic people learn more quickly when benefitting others, and their sgACC response is the most selective for prosocial learning. We thus reveal a computational mechanism driving prosocial learning in humans. This framework could provide insights into atypical prosocial behavior in those with disorders of social cognition.reinforcement learning theory | prosocial behavior | empathy | reward | subgenual anterior cingulate cortex P rosocial behaviors, namely, social behaviors or actions intended to benefit others, are a fundamental but poorly understood aspect of social interaction (1). To behave prosocially, animals need to learn about the consequences that their actions can have for others. In reinforcement learning theory (RLT), prediction errors (PEs)-differences between expected and actual outcomesdrive learning (2). RLT provides a powerful framework for understanding how animals learn to obtain rewards for themselves (3). However, the processes by which animals learn to make choices that benefit others are unknown. Here we use RLT to characterize prosocial learning, combining functional magnetic resonance imaging (fMRI) and detailed computational modeling of behavior.Studies using economic games, moral judgments, or charity donation tasks have consistently reported activity in the ventral striatum, posterior regions of the subgenual cingulate cortex/basal forebrain (hereinafter referred to as sgACC), dorsal anterior cingulate cortex (dACC), and dorsolateral prefrontal cortex (DLPFC) during prosocial behavior (4-7). Each of these regions receives input from midbrain dopaminergic neurons (8), and these cortical regions all project to the ventral ...

show abstract

Section: Methodsmentioning

confidence: 99%

“…The ventral striatum, sgACC, dACC, and DLPFC are also implicated in processing information about rewards others will receive (12-16), PEs when interacting with others (13,(17)(18)(19), and prosocial behavior (e.g., refs. 4-7 for reviews).…”

mentioning

confidence: 99%

Neurocomputational mechanisms of prosocial learning and links to empathy

Lockwood

Apps

Valton

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

186

236

View full text Add to dashboard Cite

show abstract

“…For instance, observing that others receive painful shocks following a visual stimulus can lead to conditioned fear responses and amygdala activation to this visual stimulus, even though participants never experienced the shocks themselves 71 . For positive values, several further studies also suggested that vicarious learning may follow computational principles that are similar to those for learning through personal experience [72][73][74][75] . For example, one recent study induced prediction errors at the outcome stage of altruistic choices by unexpectedly changing the outcome, either for the…”

Section: Vicarious Decision Makingmentioning

confidence: 99%

“…These studies thus suggest that both experience-based and vicarious learning of anticipated values have the same neural substrate (which supports the common currency schema). However, two other studies suggest that vicarious social learning also draws on distinct prediction-error computations in different neural structures 74,75 . These studies showed both vicarious reward prediction errors in the vmPFC (FIG.…”

Section: Prisoner's Dilemma Gamementioning

confidence: 99%

“…These studies showed both vicarious reward prediction errors in the vmPFC (FIG. 2d) and action-prediction errors in either the dorsolateral PFC (dlPFC) 74 or the dmPFC 75 that encoded violations of expectations about which action the other participant would choose. These signals occurred in addition to BOLD signals in the ventral striatum that reflected the prediction-error signal for the participant's own expected rewards 74 .…”

Section: Prisoner's Dilemma Gamementioning

confidence: 99%

See 1 more Smart Citation

The neurobiology of rewards and values in social decision making

2014

View full text Add to dashboard Cite

Of all the decisions in your life, which were the most important? Chances are high that many of them had a social component, be it when deciding about others (for example, whether to marry someone), for others (for example, which school to send your child to) or jointly with others (for example, whether to buy a house with your partner). Pathological changes specifically in the social aspects of decision making are pervasive in many psychiatric 1 and neurological 2 disorders and can have devastating consequences for individual and collective welfare. A good understanding of the neural basis of social decision making is thus an important aim of scientists in many disciplines, from basic neuroscience to medicine, psychology and economics.The neural mechanisms underlying social and nonsocial types of choices have traditionally been examined from different theoretical angles. In studies of non-social choices (for example, purchasing decisions), researchers have often focused on neural value computations associated with the rewarding properties of the choice options 3,4 . On the basis of this research, several brain structures have been proposed as elements of a brain valuation system (BOX 1). The neural value signals in these structures are thought to represent a 'common currency' for assessing the motivational relevance of all possible stimuli or actions, which is essential for guiding decision making across varying contexts 5,6 . By contrast, researchers studying social decisions have traditionally paid less attention to motivational processes, such as reward and value coding; instead, they have focused on identifying neurocognitive processes that may have uniquely evolved to guide social behaviour 7 . This emphasis on 'social-specific cognition' has identified, for example, that different visual areas seem to be functionally specialized for the perception of faces or bodies 8 and that the dorsomedial prefrontal cortex (dmPFC) and the temporoparietal junction (TPJ) may be specifically involved in representing the intentions, emotions or actions of other people [9][10][11] . The idea of 'social-specific cognition' thus implies that the control of social and non-social behaviour should fundamentally differ in terms of neural architecture and information processing demands 12 . However, this view is challenged by recent findings that social decisions may also draw on value-related brain processes that strongly resemble those seen during non-social decisions. This apparent overlap in neural representations of motivational relevance for social and non-social choices is often taken to implicitly extend the 'common currency' idea to decisions based on social factors. This proposalthat choices in both social and non-social situations are steered by identical neural value computations -therefore questions whether the motivational control of social behaviour requires dedicated neural processes.In this Review, we propose a framework for studying the possible neurobiological links in the motivational control of social...

show abstract