No abstract

Optimal behavior requires striking a balance between exploiting tried-and-true options or exploring new possibilities. Neuroimaging studies have identified different brain regions in humans where neural activity is correlated with exploratory or exploitative behavior, but it is unclear whether this activity directly implements these choices or simply reflects a byproduct of the behavior. Moreover, it remains unknown whether arbitrating between exploration and exploitation can be influenced with exogenous methods, such as brain stimulation. In our study, we addressed these questions by selectively upregulating and downregulating neuronal excitability with anodal or cathodal transcranial direct current stimulation over right frontopolar cortex during a reward-learning task. This caused participants to make slower, more exploratory or faster, more exploitative decisions, respectively. Bayesian computational modeling revealed that stimulation affected how much participants took both expected and obtained rewards into account when choosing to exploit or explore: Cathodal stimulation resulted in an increased focus on the option expected to yield the highest payout, whereas anodal stimulation led to choices that were less influenced by anticipated payoff magnitudes and were more driven by recent negative reward prediction errors. These findings suggest that exploration is triggered by a neural mechanism that is sensitive to prior less-than-expected choice outcomes and thus pushes people to seek out alternative courses of action. Together, our findings establish a parsimonious neurobiological mechanism that causes exploration and exploitation, and they provide new insights into the choice features used by this mechanism to direct decision-making.

External circumstances and internal bodily states often change and require organisms to flexibly adapt valuation processes to select the optimal action in a given context. Here, we investigate the neurobiology of context-dependent valuation in 22 human subjects using functional magnetic resonance imaging. Subjects made binary choices between visual stimuli with three attributes (shape, color, and pattern) that were associated with monetary values. Context changes required subjects to deviate from the default shape valuation and to integrate a second attribute to comply with the goal to maximize rewards. Critically, this binary choice task did not involve any conflict between opposing monetary, temporal, or social preferences. We tested the hypothesis that interactions between regions of dorsolateral prefrontal cortex (dlPFC) and ventromedial prefrontal cortex (vmPFC) implicated in self-control choices would also underlie the more general function of context-dependent valuation. Consistent with this idea, we found that the degree to which stimulus attributes were reflected in vmPFC activity varied as a function of context. In addition, activity in dlPFC increased when context changes required a reweighting of stimulus attribute values. Moreover, the strength of the functional connectivity between dlPFC and vmPFC was associated with the degree of context-specific attribute valuation in vmPFC at the time of choice. Our findings suggest that functional interactions between dlPFC and vmPFC are a key aspect of context-dependent valuation and that the role of this network during choices that require self-control to adjudicate between competing outcome preferences is a specific application of this more general neural mechanism.