Visual stimuli can acquire positive or negative value through their association with rewards and punishments, a process called reinforcement learning. Although we now know a great deal about how the brain analyses visual information, we know little about how visual representations become linked with values. To study this process, we turned to the amygdala, a brain structure implicated in reinforcement learning 1-5 . We recorded the activity of individual amygdala neurons in monkeys while abstract images acquired either positive or negative value through conditioning. After monkeys had learned the initial associations, we reversed image value assignments. We examined neural responses in relation to these reversals in order to estimate the relative contribution to neural activity of the sensory properties of images and their conditioned values. Here we show that changes in the values of images modulate neural activity, and that this modulation occurs rapidly enough to account for, and correlates with, monkeys' learning. Furthermore, distinct populations of neurons encode the positive and negative values of visual stimuli. Behavioural and physiological responses to visual stimuli may therefore be based in part on the plastic representation of value provided by the amygdala.The complex anatomical connections of the amygdala, a collection of nuclei located deep in the medial temporal lobe, make it a prime candidate for providing a representation of the value of visual stimuli. The amygdala receives inputs from the visual system and from other sensory systems that represent reinforcing stimuli 6-8 . In addition, the amygdala is likely to receive error signals that represent stimuli in relation to expectations and that may be essential in creating an updated representation of value. The source of error signals for aversive learning has not been identified; however, midbrain dopamine neurons might supply such error signals for appetitive learning 9 . These signals could influence amygdala neural responses either directly 7 or indirectly through other brain structures such as the prefrontal cortex 6,10 . The convergence of information from each of these input pathways, perhaps combined with processing that occurs through intrinsic connections within the amygdala, may form a representation of visual stimulus value.Correspondence and requests for materials should be addressed to C.D.S. (cds2005@columbia.edu). * These authors contributed equally to this work.Author Contributions J.J.P. and M.A.B. performed all experiments and conducted data analyses. S.E.M. performed some of the data analyses and contributed to many discussions. Experiments were designed and implemented in the laboratory of C.D.S. Unlike the anatomy of the amygdala, the physiological properties of amygdala neuronsespecially with respect to learning the value of visual stimuli-remain poorly understood. We therefore recorded the activity of single amygdala neurons while monkeys learned the positive or negative value of new, abstract images during a tr...

Animals and humans learn to approach and acquire pleasant stimuli and to avoid or defend against aversive ones. However, both pleasant and aversive stimuli can elicit arousal and attention, and their salience or intensity increases when they occur by surprise. Thus, adaptive behavior may require that neural circuits compute both stimulus valence--or value--and intensity. To explore how these computations may be implemented, we examined neural responses in the primate amygdala to unexpected reinforcement during learning. Many amygdala neurons responded differently to reinforcement depending upon whether or not it was expected. In some neurons, this modulation occurred only for rewards or aversive stimuli, but not both. In other neurons, expectation similarly modulated responses to both rewards and punishments. These different neuronal populations may subserve two sorts of processes mediated by the amygdala: those activated by surprising reinforcements of both valences-such as enhanced arousal and attention-and those that are valence-specific, such as fear or reward-seeking behavior.

As an organism interacts with the world, how good or bad things are at the moment, the value of the current state of the organism, is an important parameter that is likely to be encoded in the brain. As the environment changes and new stimuli appear, estimates of state value must be updated to support appropriate responses and learning. Indeed, many models of reinforcement learning posit representations of state value. We examined how the brain mediates this process by recording amygdala neural activity while monkeys performed a trace-conditioning task requiring fixation. The presentation of different stimuli induced state transitions; these stimuli included unconditioned stimuli (USs) (liquid rewards and aversive air puffs), newly learned reinforcement-predictive visual stimuli [conditioned stimuli (CSs)], and familiar stimuli long associated with reinforcement [fixation point (FP)]. The FP had a positive value to monkeys, because they chose to foveate it to initiate trials. Different populations of amygdala neurons tracked the positive or negative value of the current state, regardless of whether state transitions were caused by the FP, CSs, or USs. Positive value-coding neurons increased their firing during the fixation interval and fired more strongly after rewarded CSs and rewards than after punished CSs and air puffs. Negative value-coding neurons did the opposite, decreasing their firing during the fixation interval and firing more strongly after punished CSs and air puffs than after rewarded CSs and rewards. This representation of state value could underlie how the amygdala helps coordinate cognitive, emotional, and behavioral responses depending on the value of one's state.

Recent electrophysiological studies on the primate amygdala have advanced our understanding of how individual neurons encode information relevant to emotional processes, but it remains unclear how these neurons are functionally and anatomically organized. To address this, we analyzed cross-correlograms of amygdala spike trains recorded during a task in which monkeys learned to associate novel images with rewarding and aversive outcomes. Using this task, we have recently described two populations of amygdala neurons: one that responds more strongly to images predicting reward (positive value-coding), and another that responds more strongly to images predicting an aversive stimulus (negative value-coding). Here we report that these neural populations are organized into distinct, but anatomically intermingled, appetitive and aversive functional circuits, which are dynamically modulated as animals used the images to predict outcomes. Furthermore, we report that responses to sensory stimuli are prevalent in the lateral amygdala, and are also prevalent in the medial amygdala for sensory stimuli that are emotionally significant. The circuits identified here could potentially mediate valence-specific emotional behaviors thought to involve the amygdala.

The amygdala and orbitofrontal cortex (OFC) are often thought of as components of a neural circuit that assigns affective significance-or value-to sensory stimuli so as to anticipate future events and adjust behavioral and physiological responses. Much recent work has been aimed at understanding the distinct contributions of the amygdala and OFC to these processes, but a detailed understanding of the physiological mechanisms underlying learning about value remains lacking. To gain insight into these processes, we have focused initially on characterizing the neural signals of the primate amygdala, and more recently of the primate OFC, during appetitive and aversive reinforcement learning procedures. We have employed a classical conditioning procedure whereby monkeys form associations between visual stimuli and rewards or aversive stimuli. After learning these initial associations, we reverse the stimulus-reinforcement contingencies, and monkeys learn these new associations. We have discovered that separate populations of neurons in the amygdala represent the positive and negative value of conditioned visual stimuli. This representation of value updates rapidly upon image value reversal, as fast as monkeys learn, often within a single trial. We suggest that representations of value in the amygdala may change through multiple interrelated mechanisms: some that arise from fairly simple Hebbian processes, and others that may involve gated inputs from other brain areas, such as the OFC. Keywordsamygdala; OFC; orbitofrontal cortex; reinforcement learning; conditioning; learning; reward; aversive; value; monkey EMOTION, VALUATION, AND REINFORCEMENT LEARNINGIn humans, the regulation of emotion is extremely flexible, adapting to different sensory cues, social situations, and cognitive operations, such as the application of rules. How does the brain mediate these different aspects of emotional processing? Most prior efforts to understand emotion at the neural level have employed rodents, often using fear-conditioning and related behavioral paradigms. In human and non-human primates, however, a more flexible control of emotion is thought to be conferred by interactions between the amygdala and prefrontal cortex (PFC). 1 Indeed, in primates, as compared to non-primates, there is an extensive elaboration of the PFC and its connections with the amygdala. 2 In addition, because the visual system is a dominant sensory modality in primates, there are dense connections among the amygdala, PFC, and visual system. Thus, although many aspects of the function and organization of the amygdala and interconnected structures are conserved across species, amygdala function in primates likely expands upon and differs from processing in rodents in significant ways. For these reasons, it is important to elucidate the complex neural circuitry that regulates emotion in the rhesus monkey. Their rich behavioral and cognitive repertoire One way to approach these questions is to exploit different conditioning procedures developed by experim...