Exploration Disrupts Choice-Predictive Signals and Alters Dynamics in Prefrontal Cortex

Abstract

In uncertain environments, decision-makers must balance two goals: they must "exploit" rewarding options but also "explore" in order to discover rewarding alternatives. Exploring and exploiting necessarily change how the brain responds to identical stimuli, but little is known about how these states, and transitions between them, change how the brain transforms sensory information into action. To address this question, we recorded neural activity in a prefrontal sensorimotor area while monkeys naturally switched between exploring and exploiting rewarding options. We found that exploration profoundly reduced spatially selective, choice-predictive activity in single neurons and delayed choice-predictive population dynamics. At the same time, reward learning was increased in brain and behavior. These results indicate that exploration is related to sudden disruptions in prefrontal sensorimotor control and rapid, reward-dependent reorganization of control dynamics. This may facilitate discovery through trial and error.

Prefrontal attentional saccades explore space rhythmically

“…show that this PFC attentional spotlight explores space serially both at relevant (cued) and irrelevant (un-381 cued) locations, alternating between the exploitation and the exploration of the visual scene 25 . The activity 382 of the parietal 10 and PFC 49 cortical regions has been shown to change drastically between exploitation 383 and exploration behavior. In particular, exploration is associated with faster though less accurate 384 oculomotor behavior 10 and a disruption of PFC control signals 49 .…”

“…The activity 382 of the parietal 10 and PFC 49 cortical regions has been shown to change drastically between exploitation 383 and exploration behavior. In particular, exploration is associated with faster though less accurate 384 oculomotor behavior 10 and a disruption of PFC control signals 49 . This is proposed to facilitate the 385 processing of unexpected external events 10 , the expression of novel behavior and learning through trial 386 and error 49 .…”

Response outcomes gate the impact of expectations on perceptual decisions

“…However, there are circumstances where subjects seem able to quickly and flexibly modulate the impact of prior expectations in driving their choices. One of such examples is the switch between (1) exploiting choices which, according to their current statistical model of the environment, are more likely to yield reward and (2) exploring alternative choices that are not aimed to maximize reward given that internal model, but to reduce environmental uncertainty and eventually refine the current model (Daw et al, 2006;Ebitz et al, 2018;Karlsson et al, 2012) . In particular, when the task design potentiates stochastic exploration, rats are able to operate in an expectation-free mode in which choices did not depend on previous history (Tervo et al, 2014) .…”

Microscopic and mesoscopic effects of reward uncertainty in monkey fronto-parietal areas

“…Our finding that higher EV primarily enhanced processing within dlPFC and 7A but not across areas, suggests that, while uncertainty promotes network-wide interactions, higher rewards may promote more segregated cortical function. This interpretation may explain the recent report that FEF saccade-related activity is more focused (stronger and shorter latency) during exploitative relative to exploratory epochs 46 . It may also explain the fact that, in a task in which monkeys focused on a target location while ignoring intervening distractors, distractor-related activity was prominent in LIP but was blocked from driving spiking in the dlPFC 13 .…”