Dynamic changes in Anterior Cingulate Cortex ensembles mark the transition from exploration to exploitation

Abstract

The ability to acquire knowledge about the value of stimuli or actions factors into simple foraging behaviors as well as complex forms of decision making. The anterior cingulate cortex (ACC) is thought to be involved in these processes, although the manner in which neural representations acquire value is unclear. Here we recorded from ensembles of ACC neurons as rats learned which of 3 levers was rewarded each day through a trial and error process. Action representations remained largely stable during exploration, but there was an abrupt, coordinated and differential change in the representation of rewarded and nonrewarded levers by ACC neurons at the point where the rat realized which lever was rewarded and began to exploit it. Thus, rather than a gradual, incremental process, value learning in ACC can occur in an all-or-none manner and help to initiate strategic shifts in forging behavior.

ACC neural ensemble dynamics are structured by strategy prevalence

“…These functional rearrangements result in representational remapping, whereby ACC ensemble activity associated with a specific strategy changes markedly between behavioral contexts. This observation suggests that 'network resets' previously observed to accompany transitions to exploration (Durstewitz et al, 2010;Emberly and Jeremy, 2019;Karlsson et al, 2012) are not mere bookmarks of a behavioral state change, but rather reflect reorganizations of the ACC network designed to tag action plans with a global context-specific representation. Although 'dominant'/ 'exploratory' distinction maps well onto the 'default'/'alternative' dichotomy that has enjoyed prominence in the ACC field due to its computational appeal and explanatory power in many settings (Blanchard and Hayden, 2014;Boorman et al, 2013;Kolling et al, 2012;Procyk et al, 2000), the representational transitions in ACC likely reflect broader contextual tagging of action plans, as suggested by the observed difference in the representations associated with a specific exploratory sequence depending on the nature of the default strategy.…”

“…Specifically, we first sought to determine if ACC activity associated with a specific behavioral sequence is reorganized depending on whether that sequence represents a dominant behavioral strategy or is sampled as a part of an exploratory bout. Indeed, while abrupt changes in the activity of ACC neurons coincident with behavioral transitions to exploration have been reported previously (Durstewitz et al, 2010;Emberly and Jeremy, 2019;Karlsson et al, 2012;Powell and Redish, 2016), whether these transitions in ACC activity merely mark the behavioral state change, or reflect an actual task-related re-organization of activity whereby representations of individual strategies are also marked with contextual content, remains unclear. Targeted recordings of ACC activity -performed in a wireless configuration that did not impair the animals' behavioral flexibility -revealed that marked differences in activity associated with the execution of specific sequences in 'dominant' versus 'exploratory' contexts could indeed be readily observed across many ACC neurons, frequently evident even in non-trial averaged activity traces (Fig.…”