A neural network for information seeking

Abstract

Humans and other animals often show a strong desire to know the uncertain rewards their future has in store, even when they cannot use this information to influence the outcome. However, it is unknown how the brain predicts opportunities to gain information and motivates this information-seeking behavior. Here we show that neurons in a network of interconnected subregions of primate anterior cingulate cortex and basal ganglia predict the moment of gaining information about uncertain rewards. Spontaneous increases in their information prediction signals are followed by gaze shifts toward objects associated with resolving uncertainty, and pharmacologically disrupting this network reduces the motivation to seek information. These findings demonstrate a cortico-basal ganglia mechanism responsible for motivating actions to resolve uncertainty by seeking knowledge about the future.

The role of executive function in shaping reinforcement learning

“…By contrast, recent work has shown that uncertainty reduction and information gain are indeed reflected in neural RL computations [60]. Evidence from fMRI studies suggests that corticostriatal circuits incorporate the utility of information in reward computations, such that information is conceptualized as a reward that reinforces choices [61], even when it is not valenced [59]. The prefrontal cortex also appears to track information and uncertainty [40], which can be held in working memory to influence decision making [62] ( Figure 4).…”

“…One such example relates to the value of information. Humans are motivated to reduce uncertainty about their environment [59]. Thus, acquisition of novel information should in itself function as reinforcement.…”

Curiosity from the Perspective of Systems Neuroscience

“…Curiosity about lottery outcomes was shown to disinhibit the ACC in humans via the increased amplitude of a "feedback-related negativity" EEG signal that is believed to be related to dopaminergic projections to the ACC [68]. ACC was also shown to be responsive to anticipation of information during an observing task in monkeys [31].…”

“…Specifically, the apparent preference for advance information could be accounted for by greater levels of attentional engagement on informative trials, leading to increased associative learning for informative cues, and thus promoting their choice even in the absence of curiosity [27]. The general argument is plausible, and differential attention may explain some of the effect; however, it appears insufficient to explain several results [18,30,31]. To give one example, lateral habenula neurons reliably signal information prediction errors for informative as well as uninformative cues, indicating that animals do not forget their reward predictions due to task disengagement even in uninformative trials [32].…”

“…Though not mutually exclusive, White et al [31] proposed a cortico-basal ganglia circuit to also explain information-seeking using an observing task that was adapted from Bromberg-Martin and Hikosaka, 2011 [32], but their model showed that ACC and striatum both track uncertainty, and that the delivery of information alone (even before reward delivery) quenches this tracking activity. In addition, striatum was found to causally direct gaze towards informative targets while anterior pallidum was found to inhibit gaze to uninformative targets.…”