Reward modulation (M1) could be exploited in developing an autonomously updating brain-computer interface (BCI) based on a reinforcement learning (RL) architecture. For an autonomously updating RL-based BCI system, we would need a reward prediction error, or a state-value representation from the user’s neural activity, which the RL-BCI agent could use to update its BCI decoder. In order to understand the multifaceted effects of reward on M1 activity, we investigated how neural spiking, oscillatory activities and their functional interactions are modulated by conditioned stimuli related reward expectation. To do so, local field potentials (LFPs) and single/multi-unit activities were recorded simultaneously and bilaterally from M1 cortices while four non-human primates (NHPs) performed cued center-out reaching or grip force tasks either manually using their right arm/hand or observed passively. We found that reward expectation influenced the strength of α (8–14 Hz) power, α-γ comodulation, α spike-field coherence (SFC), and firing rates (FRs) in general in M1. Furthermore, we found that an increase in α-band power was correlated with a decrease in neural spiking activity, that FRs were highest at the trough of the α-band cycle and lowest at the peak of its cycle. These findings imply that α oscillations modulated by reward expectation have an influence on spike FR and spike timing during both reaching and grasping tasks in M1. These LFP, spike, and spike-field interactions could be used to follow the M1 neural state in order to enhance BCI decoding ( An et al., 2018 ; Zhao et al., 2018 ).

We are developing an autonomously updating brain machine interface (BMI) utilizing reinforcement learning principles. One aspect of this system is a neural critic that determines reward expectations from neural activity. This critic is then used to update a BMI decoder towards an improved performance from the user's perspective. Here we demonstrate the ability of a neural critic to classify trial reward value given activity from the primary motor cortex (M1), using neural features from single/multi units (SU/MU), and local field potentials (LFPs) with prediction accuracies up to 97% correct. A nonhuman primate subject conducted a cued center out reaching task, either manually, or observationally. The cue indicated the reward value of a trial. Features such as power spectral density (PSD) of the LFPs and spike-field coherence (SFC) between SU/MU and corresponding LFPs were calculated and used as inputs to several classifiers. We conclude that hybrid features of PSD and SFC show higher classification performance than PSD or SFC alone (accuracy was 92% for manual tasks, and 97% for observational). In the future, we will employ these hybrid features towards our autonomously updating BMI.

No abstract

Abstract-We are developing an autonomously updating brain machine interface (BMI) utilizing reinforcement learning principles. One aspect of this system is a neural critic that determines reward expectations from neural activity. This critic is then used to update a BMI decoder towards an improved performance from the user's perspective. Here we demonstrate the ability of a neural critic to classify trial reward value given activity from the primary motor cortex (M1), using neural features from single/multi units (SU/MU), and local field potentials (LFPs) with prediction accuracies up to 97% correct. A nonhuman primate subject conducted a cued center out reaching task, either manually, or observationally. The cue indicated the reward value of a trial. Features such as power spectral density (PSD) of the LFPs and spike-field coherence (SFC) between SU/MU and corresponding LFPs were calculated and used as inputs to several classifiers. We conclude that hybrid features of PSD and SFC show higher classification performance than PSD or SFC alone (accuracy was 92% for manual tasks, and 97% for observational). In the future, we will employ these hybrid features towards our autonomously updating BMI.

No abstract