Prefrontal cortex and decision making in a mixed-strategy game

Abstract

In a multi-agent environment, where the outcomes of one's actions change dynamically because they are related to the behavior of other beings, it becomes difficult to make an optimal decision about how to act. Although game theory provides normative solutions for decision making in groups, how such decision-making strategies are altered by experience is poorly understood. These adaptive processes might resemble reinforcement learning algorithms, which provide a general framework for finding optimal strategies in a dynamic environment. Here we investigated the role of prefrontal cortex (PFC) in dynamic decision making in monkeys. As in reinforcement learning, the animal's choice during a competitive game was biased by its choice and reward history, as well as by the strategies of its opponent. Furthermore, neurons in the dorsolateral prefrontal cortex (DLPFC) encoded the animal's past decisions and payoffs, as well as the conjunction between the two, providing signals necessary to update the estimates of expected reward. Thus, PFC might have a key role in optimizing decision-making strategies.

Population Response Profiles in Early Visual Cortex Are Biased in Favor of More Valuable Stimuli

“…This crossover interaction suggests that value increases the distance between the response distributions in these two populations of neurons, which should increase the probability of correctly discriminating the orientation of the stimulus (Green and Swets 1966;see also Martinez-Trujillo and Treue 2004). In turn, this increase in discriminability should promote a more rapid accumulation of sensory evidence concerning the identity of valuable stimuli in downstream decision mechanisms, almost as if the physical clarity or distinctiveness of the stimulus was enhanced (Beck et al 2008;Carrasco and McElree 2001;Carrasco et al 2004;Gold andShadlen 2002, 2007;Navalpakkam and Itti 2007;Newsome et al 1989).…”

“…However, reasonable candidates include dopaminergic (DA) neurons in the ventral tegemental area (VTA) and the substantia nigra pars compacta (SNc) that might influence activity in visual cortex via direct projections to early areas of visual cortex (Berger et al 1988(Berger et al , 1991Devoto and Flore 2006). However, these projections are generally thought to be sparse, so it is likely that indirect DA signals relayed through the striatum and then to frontal and parietal cortex play an important role in regulating value-related changes in early visual cortex (Barraclough et al 2004;Ding and Hikosaka 2006;Dorris and Glimcher 2004;Gläscher et al 2009;Glimcher 2003;Hikosaka et al 2008;Hollerman and Schultz 1998;Ikeda and Hikosaka 2003;Lau and Glimcher 2007;Leon and Shadlen 1999;Luk and Wallis 2009;Platt and Glimcher 1999;Schultz and Dickinson 2000;Seo et al 2007;Sugrue et al 2004;Wallis and Miller 2003;Watanabe 1996). Indeed, many of the cortical targets of reward signals-such as oculomotor neurons in frontal and parietal cortex-are ideally situated to send modulatory feedback signals to earlier sensory areas so that the cortical representation of high-value stimulus features can be enhanced (Bisley and Goldberg 2003;Ding and Hikosaka 2006;Gold and Shadlen 2007;Serences and Yantis 2006;Shadlen and Newsome 2001).…”

“…Here we show that feature-selective population response profiles in early visual cortex are biased in favor of stimuli deemed more likely to yield a monetary reward based on prior experience. These modulations in the population response profile may in turn increase the quality of inputs into regions of parietal and frontal cortex that integrate sensory evidence to form perceptual decisions (Beck et al 2008;Gold andShadlen 2002, 2007;Mazurek et al 2003;Roitman and Shadlen 2002;Shadlen and Newsome 2001). Future studies might investigate this relationship by systematically examining the influence of value on the ability of subjects to perform difficult visual discriminations under explicit speed pressure.…”

“…In addition, value modulates feature-selective response profiles associated with both the selected and the unselected stimuli on each trial, suggesting that simple space-and feature-based attention explanations cannot account for the present results. These modulations in response selectivity may facilitate the accumulation of information in the downstream areas that compute decisions about how and when to interact with behaviorally relevant objects in the environment (Beck et al 2008;Gold and Shadlen 2007).…”

Saccade selection when reward probability is dynamically manipulated using Markov chains

“…manual selections), we have focused on saccadic eye movements, which have been extensively studied with regard to target selection (Schall and Thompson 1999). Our current Markov task, used to demonstrate learning, is most similar to free choice experiments used to demonstrate neural correlates of subjective valuation or reward probability in the lateral intraparietal area (Platt and Glimcher 1999;Sugrue et al 2004;Dorris and Glimcher 2004;Yang and Shadlen 2007) and prefrontal cortex (Barraclough et al 2004). However, Markov chains could be easily adapted to a reaction time task in which the Markov state would determine target probability.…”