We recorded the activity of midbrain dopamine neurons in an instrumental conditioning task in which monkeys made a series of behavioral decisions on the basis of distinct reward expectations. Dopamine neurons responded to the first visual cue that appeared in each trial [conditioned stimulus (CS)] through which monkeys initiated trial for decision while expecting trial-specific reward probability and volume. The magnitude of neuronal responses to the CS was approximately proportional to reward expectations but with considerable discrepancy. In contrast, CS responses appear to represent motivational properties, because their magnitude at trials with identical reward expectation had significant negative correlation with reaction times of the animal after the CS. Dopamine neurons also responded to reinforcers that occurred after behavioral decisions, and the responses precisely encoded positive and negative reward expectation errors (REEs). The gain of coding REEs by spike frequency increased during learning act-outcome contingencies through a few months of task training, whereas coding of motivational properties remained consistent during the learning. We found that the magnitude of CS responses was positively correlated with that of reinforcers, suggesting a modulation of the effectiveness of REEs as a teaching signal by motivation. For instance, rate of learning could be faster when animals are motivated, whereas it could be slower when less motivated, even at identical REEs. Therefore, the dual correlated coding of motivation and REEs suggested the involvement of the dopamine system, both in reinforcement in more elaborate ways than currently proposed and in motivational function in reward-based decision-making and learning.
Global warming is increasingly recognized as a threat to the survival of human beings, because it could cause a serious increase in the occurrence of diseases due to environmental heat during intermittent hot weather. To assess the direct impact of extremely hot weather on human health, we investigated heat-related deaths in Japan from 1968 through 1994, analyzing the data to determine the distribution of the deaths by age and their correlation to the incidence of hot days in summer. Vital Statistics of Japan, published by the Ministry of Health and Welfare of Japan, was the source of the heat-related mortality data employed in this study. Meteorological data were obtained from the District Meteorological Observatories in Tokyo and Osaka, the two largest cities in Japan. Heat-related deaths were most prone to occur on days with a peak daily temperature above 38 degrees C, and the incidence of these deaths showed an exponential dependence on the number of hot days. Thus, even a small rise in atmospheric temperature may lead to a considerable increase in heat-related mortality, indicating the importance of combating global warming. Furthermore, half (50.1%) of the above-noted deaths occurred in children (4 years and under) and the elderly (70 years and over) irrespective of gender, indicating the vulnerability of these specific age groups to heat. Since a warmer climate is predicted in the future, the incidence of heat waves will increase, and more comprehensive measures, both medical and social, should be adopted for children of 4 years and younger the elderly to prevent heat-related deaths in these age groups.
Midbrain dopamine neurons signal reward value, their prediction error, and the salience of events. If they play a critical role in achieving specific distant goals, long-term future rewards should also be encoded as suggested in reinforcement learning theories. Here, we address this experimentally untested issue. We recorded 185 dopamine neurons in three monkeys that performed a multistep choice task in which they explored a reward target among alternatives and then exploited that knowledge to receive one or two additional rewards by choosing the same target in a set of subsequent trials. An analysis of anticipatory licking for reward water indicated that the monkeys did not anticipate an immediately expected reward in individual trials; rather, they anticipated the sum of immediate and multiple future rewards. In accordance with this behavioral observation, the dopamine responses to the start cues and reinforcer beeps reflected the expected values of the multiple future rewards and their errors, respectively. More specifically, when monkeys learned the multistep choice task over the course of several weeks, the responses of dopamine neurons encoded the sum of the immediate and expected multiple future rewards. The dopamine responses were quantitatively predicted by theoretical descriptions of the value function with time discounting in reinforcement learning. These findings demonstrate that dopamine neurons learn to encode the long-term value of multiple future rewards with distant rewards discounted.decision making | basal ganglia | temporal difference learning | primate
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.