Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

Abstract

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Reward Circuit Local Field Potential Modulations Precede Risk Taking

“…The amygdala is well-known to be strongly associated with reward outcome and reward evaluation 51,59 and prior fMRI research has suggested that it has activity changes with both positive and negative RPEs. 60 While several previous studies have identified positive RPE signals in the insula using both fMRI and intracranial EEG, 51,61 we did not identify insula RPE signals in our study. This could be due to spatial sampling of the insula in our study, as different insula regions have been previously been proposed to play different roles in reward signalling.…”

Beta and theta oscillations track effort and previous reward in the human basal ganglia and prefrontal cortex during decision making

“…Several human neuroimaging studies also report reward-selective activity in anterior regions of PFC like OFC ( 7 , 10 , 11 ) and effort-specific activity in insula and downstream regions like dorsomedial PFC that implement decisions ( 9 , 10 ), but conflicting neuroimaging evidence shows that activity in some of these regions, particularly dorsomedial PFC and BG, can respond to both reward and effort in a manner consistent with integrated value ( 7 , 8 , 11 – 13 ). These discrepancies could potentially be explained by spatial overlap between distinct local populations coding for positive and negative information within each region ( 88 , 89 ), but previous animal studies have also identified single units encoding net SV in OFC, STN, and dorsomedial PFC ( 90 – 94 ). Another possible explanation for the different functional correlates of oscillatory and spiking activity in OFC and BG is that the low frequency representations of reward and effort observed in our study may reflect afferent inputs to these regions ( 95 ), while spiking activity outputs of local computations reflect integrated net value.…”

Temporally organized representations of reward and risk in the human brain

“…However, the temporal ordering of computational representations in OFC remains to be clarified given evidence of both simultaneous 33 and sequential 34 encoding of expected value and risk. Similarly, despite evidence of RePE 35 , 36 and RiPE 4 , 17 , 37 signals in anterior insula, to date no studies have directly compared the relative timings nor tested for interactions between these variables in anterior insula.…”