Neural Signatures of Reward and Sensory Prediction Error in Motor Learning

Abstract

22Two distinct processes contribute to changes in motor commands during reach adaptation: 23 reward based learning and sensory error based learning. In sensory error based learning, the 24 mapping between sensory targets and motor commands is recalibrated according to error 25 feedback. In reward based learning, motor commands are associated with subjective value, 26 such that successful actions are reinforced. We recorded EEG from humans of either sex to 27 identify and dissociate the neural correlates of reward prediction error (RPE) and sensory 28 prediction error (SPE) during learning tasks designed to isolate each response. We designed a 29 visuomotor rotation task to isolate sensory error based learning in response to SPE, which was 30 induced by altered visuospatial feedback of hand position. In a reward learning task, we 31 isolated learning in response to RPE, which was induced by binary reward feedback. We found 32 that a fronto-central event related potential called the feedback related negativity occurred 33 specifically in response to RPE during reward based learning, while a more posterior component 34 called the P300 was associated with SPE during a visuomotor rotation task. These findings 35 reveal a dissociation between well characterized EEG signatures of error processing in two 36 distinct motor learning processes. CC-BY-NC-ND 4.0 International license peer-reviewed) is the author/funder. It is made available under aThe copyright holder for this preprint (which was not . http://dx.doi.org/10.1101/262576 doi: bioRxiv preprint first posted online Feb. 9, 2018; Page 3 Significance Statement 44Adaptation of motor output to changing task demands and dynamic environments is thought to 45 occur on the basis of distinct forms of prediction error. Reward prediction error occurs when 46 the subjective value of an outcome is different than the expected value, while sensory 47 prediction error occurs when the low-level sensory consequences of motor commands are 48 different than predicted. It remains unclear how EEG activity patterns associated with error 49 processing apply to these distinct forms of motor adaptation. We found that an event related 50 potential component called the feedback related negativity responded only to reward 51 prediction error, while the P300 component is modulated by sensory prediction error and 52 correlated to sensory error based learning.

Distinct Neural Signatures of Outcome Monitoring following Selection and Execution Errors

“…However, the fMRI investigation did show increased ACC activity in response to execution versus selection errors, suggesting that execution errors have their own neural signature. Relatedly, the FRN has been shown to vary in response to reward prediction errors but not sensory prediction errors when comparing signals from reward and visuomotor rotation learning tasks (Palidis et al, 2019). Together with the fMRI results, the present data pose the possibility that execution error processing may be distinct from dopamine-related reinforcement learning processes.…”

The feedback‐related negativity indexes prediction error in active but not observational learning

“…With respect to the ERP analyses, we expected to confirm that the FRN encodes PE in an active learning condition (Fischer & Ullsperger, ; Holroyd & Coles, ; Palidis et al, ; Walsh & Anderson, ). Therefore, we predicted a relationship between neural activity in the FRN time window and the trial‐by‐trial magnitude of model‐generated PEs.…”

“…This aspect of human behavior is formalized in Reinforcement Learning (RL) Theory, which states that the subjective value of an action is updated when there is a discrepancy between the predicted and actual outcome of an action—a prediction error (PE) (Barto & Sutton, ). PE computations are represented physiologically in electroencephalogram (EEG) signal in the form of the feedback‐related negativity (FRN), also referred to as the reward positivity (RewP) (Proudfit, ), which is an event‐ related potential (ERP) that correlates positively with PE (Fischer & Ullsperger, ; Gehring & Willoughby, ; Holroyd & Coles, ; Palidis, Cashaback, & Gribble, ; Walsh & Anderson, ) and whose likely origin is the posterior medial frontal cortex (pMFC) (Gehring & Willoughby, ; Gruendler, Ullsperger, & Huster, ; Holroyd & Coles, ; Palidis et al, ; Walsh & Anderson, ).…”

Task errors contribute to implicit remapping in sensorimotor adaptation

“…Perturbations also elicit either an unexpected failure to attain the reward of hitting the target (i.e., a negative reward prediction error) (Izawa & Shadmehr, 2011), and/or an unexpected punishment of missing the target (i.e., a positive punishment error). It is clear that behavioural responses to perturbations are affected not only by sensory prediction errors, but also by reward prediction errors (Izawa & Shadmehr, 2011; Cashaback et al , 2017; Palidis et al , 2018). However, how reward prediction errors and rewards affect sensorimotor adaptation is not fully understood.…”

“…In contrast, sensory prediction errors are known to result in implicit remapping: a change in the perceived sensory consequences of motor command (Mazzoni & Krakauer, 2006). Although adaptive behaviour to compensate for perturbations can be driven by sensory prediction errors or reward prediction errors (Izawa & Shadmehr, 2011; Nikooyan & Ahmed, 2015; Cashaback et al , 2017; Palidis et al , 2018), it has been suggested that only sensory prediction errors can produce a change in the system that predicts sensory consequences of motor commands: reward prediction errors alone are insufficient (Izawa & Shadmehr, 2011; Nikooyan & Ahmed, 2015). However, because these studies never made both sensory prediction errors and reward prediction errors concurrently available in the same conditions, it remains unclear whether reward prediction errors modulate implicit remapping resulting from sensory prediction errors.…”