Pavlovian-Instrumental Interaction in ‘Observing Behavior’

Beierholm, Ulrik; Dayan, Peter

doi:10.1371/journal.pcbi.1000903

Cited by 41 publications

(46 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One debated question is whether EIG estimates rely on explicit measures of uncertainty (37) or higher-order effects of rewards [such as convex utility function or nonstandard effects of RPE that have yet to be characterized in individual cells (38)(39)(40)]. A second key question is whether EIG is computed dynamically based on the uncertainty of each forthcoming action or relies on long-term estimates of average validity.…”

Section: Discussionmentioning

confidence: 99%

Parietal neurons encode expected gains in instrumental information

Foley

Kelly

Mhatre

et al. 2017

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

In natural behavior, animals have access to multiple sources of information, but only a few of these sources are relevant for learning and actions. Beyond choosing an appropriate action, making good decisions entails the ability to choose the relevant information, but fundamental questions remain about the brain's information sampling policies. Recent studies described the neural correlates of seeking information about a reward, but it remains unknown whether, and how, neurons encode choices of instrumental information, in contexts in which the information guides subsequent actions. Here we show that parietal cortical neurons involved in oculomotor decisions encode, before an information sampling saccade, the reduction in uncertainty that the saccade is expected to bring for a subsequent action. These responses were distinct from the neurons' visual and saccadic modulations and from signals of expected reward or reward prediction errors. Therefore, even in an instrumental context when information and reward gains are closely correlated, individual cells encode decision variables that are based on informational factors and can guide the active sampling of action-relevant cues.information sampling | saccades | attention | decisions | reward I n natural behavior, animals have access to multiple sources of information, but few of these sources are relevant for learning or action. Making good decisions therefore entails not only the selection of the ultimate action but, more primarily, the decision of which source of information to sample. Decisions about information sampling are central for tasks as diverse as making a medical diagnosis (which is the best test to prescribe?), making categorization decisions (which is the most informative feature?) (1, 2), and guiding skilled actions (what should I keep my eyes on while driving?). Despite the ubiquity and significance of active sampling mechanisms, few studies have been devoted to understanding these mechanisms and their importance for decision theories. Evidence accumulation has been extensively examined in decision research (3) but has been portrayed as a passive process, in the sense that decision makers rely on predetermined (experimenter selected) sources of information but cannot themselves determine which source to consult to guide a future action.Recent studies begin to shed light on this question by showing that animals (including pigeons, monkeys, and humans) prefer to observe cues that are predictive rather than nonpredictive about a future reward, and that the value of informative cues is encoded in the orbital frontal cortex and midbrain dopamine (DA) cells (4, 5). These investigations, however, have been limited to noninstrumental contexts in which animals seek to obtain information about a reward merely in order to know, but cannot act based on the information. Very little is known about the much more common scenario in which animals sample instrumental information to make decisions and guide future actions (6).Understanding instrumental sampling poses...

show abstract

Section: Discussionmentioning

confidence: 99%

Parietal neurons encode expected gains in instrumental information

Foley

Kelly

Mhatre

et al. 2017

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…On each trial, participants had a 50% chance that their curiosity would be satisfied by seeing the outcome (curiosity relief) and a 50% chance that the outcome was withheld. This manipulation was explicitly instructed to subjects and it uncoupled curiosity responses from the actual receipt of the outcome (thus rendering Pavlovian bias accounts of our observing behavior less likely, Beierholm and Dayan, 2010). In the follow-up fMRI experiment (Experiment 3, see below) it enabled us to investigate the neural consequences of curiosity relief (Curiosity Relief -Yes versus Curiosity Relief -No).…”

Section: Procedures -Behavioralmentioning

confidence: 99%

“…In fact, monkeys are willing to sacrifice a substantial amount (20% -33%) of primary reward to get advance information (Blanchard et al, 2015). Some researchers (Dinsmoor, 1983;Shannon & Weaver, 1949) have highlighted this phenomenon as an exemplary departure from normatively optimal (reward-guided) behavior (but see Beierholm & Dayan, 2010, for an alternative account).…”

Section: Introductionmentioning

confidence: 99%

Induction and Relief of Curiosity Elicit Parietal and Frontal Activity

Lieshout

Vandenbroucke

Müller

et al. 2017

Preprint

View full text Add to dashboard Cite

Curiosity is a basic biological drive, but little is known about its behavioral and neural mechanisms. We can be curious about several types of information. On the one hand, curiosity is a function of the expected value of information, serving primarily to help us maximize reward. On the other hand, curiosity can be a function of the uncertainty of information, helping us to update what we know. In the current studies, we aimed to disentangle the contribution of information uncertainty and expected value of rewards to curiosity in humans of either sex. To this end, we designed a lottery task in which uncertainty and expected value of trial outcomes were manipulated independently, and examined how neural activity and behavioral measures of curiosity were modulated by these factors. Curiosity increased linearly with increased outcome uncertainty, both when curiosity was explicitly probed as well as when it was implicitly tested by people's willingness to wait. Increased expected value, however, did not strongly relate to these curiosity measures. Neuroimaging results showed greater BOLD response with increasing outcome uncertainty in parietal cortex at the time of curiosity induction. Outcome updating when curiosity was relieved resulted in an increased signal in the insula, orbitofrontal cortex and parietal cortex. Furthermore, the insula showed a linear increase corresponding to the size of the information update. These results suggest that curiosity is monotonically related to the uncertainty about one's current world model, the induction and relief of which are associated with activity in parietal and insular cortices respectively. Significance statementHumans are curious by nature. When you hear your phone beep, you probably feel the urge to check the message right away, even though the message itself likely doesn't give you a direct reward. In this study, we demonstrated that curiosity can be driven by outcome uncertainty, irrespective of reward. The induction of curiosity was accompanied by increased activity in the parietal cortex, whereas the information update at the time of curiosity relief was associated with activity in insular cortex. These . CC-BY-NC-ND 4.0 International license peer-reviewed) is the author/funder. It is made available under a The copyright holder for this preprint (which was not . http://dx.doi.org/10.1101/195461 doi: bioRxiv preprint first posted online Sep. 29, 2017; 3 findings advance our understanding of the behavioral and neural underpinnings of curiosity, which lies at the core of human information-seeking and serves to optimize the individual's current world-model.

show abstract

“…3A) (Hogarth et al, 2010). It remains therefore possible that by selecting the informative cue the monkeys did not specifically seek information, but simply sought to minimize their effort (by avoiding having to lick for or look at a low-reward pattern) or perhaps to bring about the motivationally salient, high-reward pattern (Beierholm and Dayan, 2010). At this time therefore it remains an open question whether the brain has a bona fide reliability representation.…”

Section: Attention For Action: Reliability Relevance and Rewardsmentioning

confidence: 99%

Attention, Learning, and the Value of Information

Gottlieb

2012

Neuron

215

185

View full text Add to dashboard Cite

Despite many studies on selective attention, fundamental questions remain about its nature and neural mechanisms. I draw from the animal and machine learning fields that describe attention as a mechanism for active learning and uncertainty reduction, and explore the implications of this view for understanding visual attention and eye movement control. A closer integration of these different views has the potential greatly to expand our understanding of oculomotor control and our ability to use this system as a window into high level but poorly understood cognitive functions, including curiosity and exploration and inferring internal models of the external world.

show abstract

Pavlovian-Instrumental Interaction in ‘Observing Behavior’

Cited by 41 publications

References 52 publications

Parietal neurons encode expected gains in instrumental information

Parietal neurons encode expected gains in instrumental information

Induction and Relief of Curiosity Elicit Parietal and Frontal Activity

Attention, Learning, and the Value of Information

Contact Info

Product

Resources

About