Pupil size encodes uncertainty during exploration

Abstract

Exploration is an important part of decision making and is crucial to maximizing long-term reward. Past work has shown that people use different forms of uncertainty to guide exploration. In this study, we investigate the role of the pupil-linked arousal system in uncertainty-guided exploration. We measured participants’ pupil dilation (N = 48) while they performed a two- armed bandit task. Consistent with previous work, we found that people adopted a hybrid of directed, random and undirected exploration, which are sensitive to relative uncertainty, total uncertainty and value difference between options, respectively. We also found a positive correlation between pupil size and total uncertainty. Furthermore, augmenting the choice model with subject-specific total uncertainty estimates decoded from the pupil size improved predictions of held-out choices, suggesting that people used the uncertainty estimate encoded in pupil size to decide which option to explore Together, the data shed light on the computations underlying uncertainty-driven exploration. Under the assumption that pupil size reflects Locus Coeruleus-Norepinephrine (LC-NE) neuromodulatory activity, these results also extend the theory of LC-NE function in exploration, highlighting its selective role in driving uncertainty- guided random exploration.

A Causal Role of the Right Dorsolateral Prefrontal Cortex in Random Exploration

“…In recent years, there has been converging evidence showing dissociable neural implementation underlying these two explorative strategies in the human brain. The first line of research comes from studies supporting the differential role of dopamine and noradrenaline in mediating exploration [Chakroun et al, 2020, Cremer et al, 2023, Dubois et al, 2021, Fan et al, 2023]. Pharmacological intervention (manipulation) in human participants supports the hypothesis that dopamine mediates direct exploration [Chakroun et al, 2020, Cremer et al, 2023], while norepinephrine controls random exploration [Dubois et al, 2021, Fan et al, 2023].…”

“…The first line of research comes from studies supporting the differential role of dopamine and noradrenaline in mediating exploration [Chakroun et al, 2020, Cremer et al, 2023, Dubois et al, 2021, Fan et al, 2023]. Pharmacological intervention (manipulation) in human participants supports the hypothesis that dopamine mediates direct exploration [Chakroun et al, 2020, Cremer et al, 2023], while norepinephrine controls random exploration [Dubois et al, 2021, Fan et al, 2023]. The second line of research comes from studies that showed the dissociable role of the right frontopolar cortex (rFPC) and dorsolateral prefrontal cortex (rDLPFC) in controlling direct and random exploration, respectively.…”