Pupil size predicts the onset of exploration in brain and behavior

Abstract

In uncertain environments, intelligent decision-makers exploit actions that have been rewarding in the past, but also explore actions that could be even better. Several neuromodulatory systems are implicated in exploration, based, in part, on work linking exploration to pupil size-a peripheral correlate of neuromodulatory tone and index of arousal. However, pupil size could instead track variables that make exploration more likely, like volatility or reward, without directly predicting either exploration or its neural bases. Here, we simultaneously measured pupil size, exploration, and neural population activity in the prefrontal cortex while two rhesus macaques explored and exploited in a dynamic environment. We found that pupil size under constant luminance specifically predicted the onset of exploration, beyond what could be explained by reward history. Pupil size also predicted disorganized patterns of prefrontal neural activity at both the single neuron and population levels, even within periods of exploitation. Ultimately, our results support a model in which pupil-linked mechanisms promote the onset of exploration via driving the prefrontal cortex through a critical tipping point where prefrontal control dynamics become disorganized and exploratory decisions are possible.

Humans forage for reward in reinforcement learning tasks

“…RL models also do not naturally predict important nonlinearities in brain and behavior like abrupt state transitions in value-coding regions [21,24,[51][52][53]. By contrast, the foraging model predicts that there should be nonlinearities in the neural and behavioral correlates of value whenever the threshold is crossed and exploration begins-exactly when such nonlinearities are observed experimentally [21,24].…”

“…We would note that is especially important to be cautious in interpreting correlations between value and neural activity because (1) time series data are prone to spurious correlations, (2) putative neural correlates of value could be caused by other mechanisms, like mnemnonic errors [49], and (3) neural activity have a complex, nonlinear relationship with value even when value is explicitly cued [50]. RL models also do not naturally predict important nonlinearities in brain and behavior like abrupt state transitions in value-coding regions [21,24,[51][52][53]. By contrast, the foraging model predicts that there should be nonlinearities in the neural and behavioral correlates of value whenever the threshold is crossed and exploration begins-exactly when such nonlinearities are observed experimentally [21,24].…”

“…In Experiment 1, participants on the Amazon mTurk platform (258 people, 120 female, 2 other or non-reporting) performed a classic sequential decision-making task known as a restless k-armed bandit (Figure 1A; [6,[20][21][22][23][24]). The participants were asked to repeatedly choose between k options, each of which was associated with some probability of reward.…”

Persistent Decision-Making in Mice, Monkeys, and Humans

“…Exploratory decision-making is affected by physiological and psychological processes that operate in the time scale of the body, not just in the time scale of trials (Shourkeshti et al, 2023). Therefore, it is possible mice were less persistent than primates because each trial took longer in this species, compared to humans and monkeys.…”