Decoding Kinematic Information From Primary Motor Cortex Ensemble Activities Using a Deep Canonical Correlation Analysis

Abstract

The control of arm movements through intracortical brain-machine interfaces (BMIs) mainly relies on the activities of the primary motor cortex (M1) neurons and mathematical models that decode their activities. Recent research on decoding process attempts to not only improve the performance but also simultaneously understand neural and behavioral relationships. In this study, we propose an efficient decoding algorithm using a deep canonical correlation analysis (DCCA), which maximizes correlations between canonical variables with the non-linear approximation of mappings from neuronal to canonical variables via deep learning. We investigate the effectiveness of using DCCA for finding a relationship between M1 activities and kinematic information when non-human primates performed a reaching task with one arm. Then, we examine whether using neural activity representations from DCCA improves the decoding performance through linear and non-linear decoders: a linear Kalman filter (LKF) and a long short-term memory in recurrent neural networks (LSTM-RNN). We found that neural representations of M1 activities estimated by DCCA resulted in more accurate decoding of velocity than those estimated by linear canonical correlation analysis, principal component analysis, factor analysis, and linear dynamical system. Decoding with DCCA yielded better performance than decoding the original FRs using LSTM-RNN (6.6 and 16.0% improvement on average for each velocity and position, respectively; Wilcoxon rank sum test, p < 0.05). Thus, DCCA can identify the kinematics-related canonical variables of M1 activities, thus improving the decoding performance. Our results may help advance the design of decoding models for intracortical BMIs.

Where is all the nonlinearity: flexible nonlinear modeling of behaviorally relevant neural dynamics using recurrent neural networks

“…In addition, unlike what the second stage of RNN PSID enables, these prior works do not model additional neural dynamics beyond those that decode behavior, and thus do not aim to dissociate the two types of neural dynamics. Moreover, each of these works 15,16 , including those with non-causal sequential autoencoders 2 , use specific nonlinear RNN structures whereas in RNN PSID the nonlinear structure is automatically selected in a way that best suits the training data within an inner cross-validation (Methods). Finally, importantly, here unlike prior works, we further dissect the nonlinearity and explore how one can isolate the nonlinearity in specific parameters of the nonlinear RNN, each of which has interpretable roles.…”

“…In equation (16), each multiplication between a model parameter and a vector (e.g. ′ ) can be thought of as a multi-input-multi-output linear function applied to an input vector (e.g.…”

“…RNNs provide an especially potent architecture for building nonlinear dynamic models 2,15,16,19 . A major part of the appeal of RNNs is that regardless of the computational graph of the model, model parameters can always be learned from the training data using general numerical optimization approaches 20 , which can enable easy exploration of different model structures.…”

“…Prior dynamic models of neural population dynamics have often been linear or generalized linear 1,4,[9][10][11][12] . Thus, recently there has been growing interest toward models that support piece-wise linear 13 , switching linear 5,14 , or nonlinear 2,[15][16][17][18] neural dynamics, especially in applications such as single trial smoothing of neural population activity 2 and decoding behavior 15,16,18 . However, two major challenges have remained unaddressed in modeling the dynamical transformation of neural activity to behavior.…”

Finding Kinematics-Driven Latent Neural States From Neuronal Population Activity for Motor Decoding

“…Meanwhile, DCCA, which finds optimal parameters to maximize the correlation between latent variables for two random variables is often used for different purposes other than MLP or LSTM [27]. Kim et al showed the effects of latent variables extracted by DCCA on motor decoding performance, where the two random variables correspond to firing rates and kinematic variables [29]. This examined the feasibility of motor decoding with DCCA structure-extracting latent variables by directly associating neuronal population activity with kinematic variables.…”