The rodent lateral orbitofrontal cortex as an arbitrator selecting between model-based and model-free learning systems.

Abstract

Our understanding of orbitofrontal cortex (OFC) function has progressed remarkably over the past decades in part due to theoretical advances in associative and reinforcement learning theories. These theoretical accounts of OFC function have implicated the region in progressively more psychologically refined processes from the value and sensory specific properties of expected outcomes to the representation and inference over latent state representations in cognitive maps of task space. While these accounts have been successful at modelling many of the effects of causal manipulation of OFC function in both rodents and primates, recent findings suggest that further refinement of our current models are still required. Here we briefly review how our understanding of OFC function has developed to understand two cardinal deficits following OFC dysfunction: reversal learning and outcome devaluation. We then consider recent findings that OFC dysfunction also significantly affects initial acquisition learning, often assumed to be intact. To account for these findings, we consider a possible role for the OFC in the arbitration and exploration between model-free and model-based learning systems, off-line updating of model-based representations. While the function of the OFC as a whole is still likely to be integral to the formation and use of a cognitive map of task space, these refinements suggest a way in which distinct orbital subregions, such as the rodent lateral OFC, might contribute to this overall function.

Lateral Orbitofrontal Cortex and Basolateral Amygdala Regulate Sensitivity to Delayed Punishment during Decision-Making

“…It is possible that rats performing DPDT are not discounting delayed punishment but are instead unaware of impending punishment because of reduced temporal contiguity between action and outcome, leading to increased choice of options with delayed punishment. OFC has a well-established role in outcome representation ( Ursu and Carter, 2005 ; Mainen and Kepecs, 2009 ; Panayi et al, 2021 ); therefore, if choice of delayed punishment was driven by reduced punishment expectancy, inactivation of LOFC would further disrupt expectancy and increase choice of delayed punishment. However, LOFC inactivation here had the opposite effect, reducing choice of rewards with delayed punishment.…”

The orbitofrontal cartographer.

“…Yet the precise content of these predictions and the degree to which OFC contributes to agency contingent upon them has become contentious, with several plausible theories advocating different answers to these questions. These theories range from suggestions that OFC is important for tracking the associative relationships between stimuli (Rolls, 1996; Schoenbaum et al, 2009), to assigning credit (Noonan et al, 2010), to computing the value of options during economic decisions (Padoa-Schioppa, 2011; Padoa-Schioppa & Conen, 2017), to contributing to cognitive mapping (Wilson et al, 2014), or even as arbitrating between model-free or model-based approaches (Panayi et al, 2021). Although these theories overlap in important ways, such as their reliance on the use of models of the environment, they make substantially different claims as to the specific function of the OFC.…”

Distinct roles of monkey OFC-subcortical pathways in adaptive behavior

“…This idea has been supported by a number of recording and lesion studies, which have consistently demonstrated the involvement of the OFC in such learning processes(35, 36). However, other recent reports suggest that rodent OFC, or the ventral PFC in humans, is not simply engaged in either experience- or inference-based value-updating strategies, but rather in regulating the switch between them according to current contexts(10, 37). Our present results are consistent with and extend this idea; we can now assign distinct roles for at least two different OFC projections to subcortical structures (rmCD and MDm).…”