Decoding neural signals and discovering their representations with a compact and interpretable convolutional neural network

Abstract

Brain-computer interfaces (BCIs) decode information from neural activity and send it to external 1 devices. In recent years, we have seen an emergence of new algorithms for BCI decoding. Here 2 we propose a compact architecture for adaptive decoding of electrocorticographic (ECoG) data into 3 finger kinematics. We also describe a theoretically justified approach to interpreting the spatial 4 and temporal weights in the architectures that combine adaptation in both space and time, such as 5 ours. In hese architectures the weights are optimized not only for decoding of target signals but 6 also for tuning away from the interfering sources, in both the spatial and the frequency domains. 7 When applied to a dataset taken from the repository of Berlin BCI IV competition, our architecture 8 outperformed the competition winners without the need for feature selection. Moreover, by looking 9 at the architecture weights we could explain in physiological terms how our algorithm decodes 10 spatial and temporal parameters of finger kinematics. As such, the proposed architecture offers a 11 good decoder and a tool for investigating neural mechanisms of motor control. 15Brain-computer interfaces (BCIs) link the nervous system to external devices [4] or even the other brains [15]. While 16 there exist many applications of BCIs [1], clinically relevant BCIs have received most attention that aid in rehabilitation 17 of patients with sensory, motor, and cognitive disabilities [12]. Clinical uses of BCIs range from assistive devices to 18 neural prostheses that restore functions abolished by neural trauma or disease [2]. 19 BCIs can deal with a variety of neural signals [14, 8] such as, for example, electroencephalographic (EEG) potentials 20 sampled with electrodes placed on the surface of the head [11], or neural activity recorded invasively with the elec-21 trodes implanted in the cortex [6] or places onto the cortical surface [18]. The latter method, which we consider here, 22is called electrocorticography (ECoG ). Accurate decoding of neural signals is key to building efficient BCIs. 23A PREPRINT -JUNE 2, 2020 BCI signal processing comprises several steps, including signal conditioning, feature extraction, and decoding. In 24 the modern machine-learning algorithms, feature extraction and decoding are not separate but rather simultaneous 25 computations performed with the computational architectures called Deep Neural Networks (DNN) [9]. DNNs de-26 rive features automatically when executing regression or classification tasks. While it is often difficult to interpret 27 the computations performed by a DNN, such interpretations are essential to gain understanding of the properties of 28 brain activity contributing to decoding, and to ensure that artifacts do not affect the decoding results. In particular, 29 interpretation of features computed by the first several layers of a DNN could shed light on the neurophysiological 30 mechanisms underlying the behavior being studied. Ideally, by examining DNN weights, one should be able...

Intracortical Hindlimb Brain–Computer Interface Systems: A Systematic Review

“…Recently, researchers reported that using deep neural networks (DNN) for decoding can improve decoding accuracy. Studies have used Restricted Boltzmann machine [126], LSTM [127]- [129], Convolutional Neural Network (CNN) [130], [131], temporal convolutional network (TCN) [132], recurrent neural network (RNN), QuasiRNN [133], multiplicative RNN (MRNN) [134], gated recurrent unit network (GRU) [135], and combination of CNN and LSTM [118], [136] for discrete and continuous movement-related parameters decoding. DNN models need to be trained by large-scale datasets in order to avoid overfitting and obtain high accuracies [137], [138].…”

Transformer-Based Methods for Neural Decoding

“…Among them LSTMs are the most commonly used because they are able to learn long-range dependencies better than other recurrent structures [1][2][3][4][5][6]. Convolutional Neural Networks (CNNs) are also frequently used for decoding neural signals in the form of fMRI image, calcium image or multi-channel EEG waves [7][8][9] because they are able to learn local dependencies of the data. Although most of these deep learning algorithms have achieved better performances compared with traditional machine learning methods, they still suffer from problems such as gradient vanishing and the weakness of extracting global features.…”