Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning

Abstract

Objective. Brain-machine interfaces (BMIs) seek to restore lost motor functions in individuals with neurological disorders by enabling them to control external devices directly with their thoughts. This work aims to improve robustness and decoding accuracy that currently become major challenges in the clinical translation of intracortical BMIs. Approach. We propose entire spiking activity (ESA)-an envelope of spiking activity that can be extracted by a simple, threshold-less, and automated technique-as the input signal. We couple ESA with deep learning-based decoding algorithm that uses quasi-recurrent neural network (QRNN) architecture. We evaluate comprehensively the performance of ESA-driven QRNN decoder for decoding hand kinematics from neural signals chronically recorded from the primary motor cortex area of three non-human primates performing different tasks. Main results. Our proposed method yields consistently higher decoding performance than any other combinations of the input signal and decoding algorithm previously reported across long-term recording sessions. It can sustain high decoding performance even when removing spikes from the raw signals, when using the different number of channels, and when using a smaller amount of training data. Significance. Overall results demonstrate exceptionally high decoding accuracy and chronic robustness, which is highly desirable given it is an unresolved challenge in BMIs.

Selection of Essential Neural Activity Timesteps for Intracortical Brain–Computer Interface Based on Recurrent Neural Network

“…Thus, we evaluated the effect of timestep selection for three RNN-based neural decoders. Furthermore, we compared the decoding performance with state-of-the-art methods including recurrent exponential-family harmonium (rEFH) [25] for the monkey Indy data and the entire spiking activity-driven quasi-RNN (ESA-driven QRNN) [8] for the monkey N. Figure 6a-c show that when the number of timesteps was reduced from T to T * , all the neural decoders reduced their performance for monkey Indy. However, the decoding performances of both the LSTM and GRU (R 2 = 0.74 ± 0.05 for LSTM and R 2 = 0.76 ± 0.05 for GRU) were significantly better than that achieved by the rEFH in [25] (p < 0.001 for T = 10, 15, 20, Friedman ANOVA test, followed by the Wilcoxon signed-rank test).…”

“…We evaluated the decoding performance using the coefficient of determination (R 2 ) [25] and Pearson's correlation coefficient (CC) [8]. The coefficient of determination measures the goodness of fit of a neural decoder as follows:…”

“…For monkey Indy, the proposed TTS was compared with the state-of-the-art method in [25] that uses the coefficient of determination. For monkey N, the proposed TTS was compared with state-of-the-art method in [8] that uses Pearson's correlation coefficient.…”

“…The experiments were implemented using the PyTorch library and executed on a computer equipped with an NVIDIA GeForce GTX 1080 Ti GPU. The training and testing protocols for the monkey Indy and monkey N data were the same as those used in [25] and [8], respectively. For monkey Indy, the first 5000 samples of each session were used to establish the training set, and the remaining samples were used as the test set.…”

“…Thus, the length of the neural sequence, a hyperparameter for decoding, is usually designed manually by trial and error. In the literature, various terminologies are used for the length of a neural sequence, including number of timesteps [8,9], number of taps [6,10], length of the sliding window [4] and time window size [7,11]. We refer to the length of a neural sequence for decoding as the number of timesteps for consistency with most neural decoders and considering its use in neuroscience [12].…”