Necessary Contributions of Human Frontal Lobe Subregions to Reward Learning in a Dynamic, Multidimensional Environment

Abstract

Real-world decisions are typically made between options that vary along multiple dimensions, requiring prioritization of the important dimensions to support optimal choice. Learning in this setting depends on attributing decision outcomes to the dimensions with predictive relevance rather than to dimensions that are irrelevant and nonpredictive. This attribution problem is computationally challenging, and likely requires an interplay between selective attention and reward learning. Both these processes have been separately linked to the prefrontal cortex, but little is known about how they combine to support learning the reward value of multidimensional stimuli. Here, we examined the necessary contributions of frontal lobe subregions in attributing feedback to relevant and irrelevant dimensions on a trial-by-trial basis in humans. Patients with focal frontal lobe damage completed a demanding reward learning task where options varied on three dimensions, only one of which predicted reward. Participants with left lateral frontal lobe damage attributed rewards to irrelevant dimensions, rather than the relevant dimension. Damage to the ventromedial frontal lobe also impaired learning about the relevant dimension, but did not increase reward attribution to irrelevant dimensions. The results argue for distinct roles for these two regions in learning the value of multidimensional decision options under dynamic conditions, with the lateral frontal lobe required for selecting the relevant dimension to associate with reward, and the ventromedial frontal lobe required to learn the reward association itself.

Specialized Representations of Value in the Orbital and Ventrolateral Prefrontal Cortex: Desirability versus Availability of Outcomes

“…Accordingly, the impairment we observed on the 3-choice probabilistic learning task could result from a deficit in attentional selection, in learning, or in some combination of the two. A recent study in humans supports the idea of that VLPFC plays a role in attentional selection (Vaidya and Fellows, 2016). …”

A Critical Role for Human Ventromedial Frontal Lobe in Value Comparison of Complex Objects Based on Attribute Configuration

“…The task also had limited power to assess elemental multiattribute choices requiring trade-offs, limiting con-clusions about whether the VMF is also involved under those conditions. Interestingly, we found preliminary evidence that patients with damage affecting other frontal regions had difficulty with such trials, perhaps reflecting the role of the lateral and dorsomedial prefrontal cortices in attentional set-shifting (Dias et al, 1996;Vaidya and Fellows, 2016). Further work on the prefrontal mechanisms of individual attribute-value trade-offs in multiattribute choice is needed.…”

Ventromedial prefrontal cortex compression during concept learning

“…Our hypothesized view of vmPFC function is based on SUS-TAIN's formalism of highly interactive mechanisms of selective attention and learning 8 , functions theoretically mapped onto interactions between PFC and the hippocampus 9,28,32 . Support for this view is found in recent patient work that has demonstrated a causal link between attentional processes and vmPFC function in decision making [33][34][35] . These studies have shown that lesions to vmPFC disrupt attentional guidance based on prior experience with cue-reward associations 35 , learning the value of task-diagnostic features during probabilistic learning 33 , and value comparison during reinforcement learning 34 .…”

“…Support for this view is found in recent patient work that has demonstrated a causal link between attentional processes and vmPFC function in decision making [33][34][35] . These studies have shown that lesions to vmPFC disrupt attentional guidance based on prior experience with cue-reward associations 35 , learning the value of task-diagnostic features during probabilistic learning 33 , and value comparison during reinforcement learning 34 . These findings have been recently extended to healthy humans in a neuroimaging study which demonstrated that value signals in vmPFC are dynamically biased by attention during reinforcement learning 13 .…”