Motor imagery electroencephalography (MI-EEG), which is an important subfield of active brain-computer interface (BCI) systems, can be applied to help disabled people to consciously and directly control prosthesis or external devices, aiding them in certain daily activities. However, the low signal-tonoise ratio and spatial resolution make MI-EEG decoding a challenging task. Recently, some deep neural approaches have shown good improvements over state-of-the-art BCI methods. In this study, an end-to-end scheme that includes a multi-layer convolution neural network is constructed for an accurate spatial representation of multi-channel grouped MI-EEG signals, which is employed to extract the useful information present in a multi-channel MI signal. Then the invariant spatial representations are captured from across-subjects training for enhancing the generalization capability through a stacked sparse autoencoder framework, which is inspired by representative deep learning models. Furthermore, a quantitative experimental analysis is conducted on our private dataset and on a public BCI competition dataset. The results show the effectiveness and significance of the proposed methodology.INDEX TERMS brain-computer interface, discriminative and representative deep learning, feature fusion, convolution neural network, stacked sparse autoencoder

The effective decoding of motor imagination EEG signals depends on significant temporal, spatial, and frequency features. For example, the motor imagination of the single limbs is embodied in the μ (8–13 Hz) rhythm and β (13–30 Hz) rhythm in frequency features. However, the significant temporal features are not necessarily manifested in the whole motor imagination process. This paper proposes a Multi-Time and Frequency band Common Space Pattern (MTF-CSP)-based feature extraction and EEG decoding method. The MTF-CSP learns effective motor imagination features from a weak Electroencephalogram (EEG), extracts the most effective time and frequency features, and identifies the motor imagination patterns. Specifically, multiple sliding window signals are cropped from the original signals. The multi-frequency band Common Space Pattern (CSP) features extracted from each sliding window signal are fed into multiple Support Vector Machine (SVM) classifiers with the same parameters. The Effective Duration (ED) algorithm and the Average Score (AS) algorithm are proposed to identify the recognition results of multiple time windows. The proposed method is trained and evaluated on the EEG data of nine subjects in the 2008 BCI-2a competition dataset, including a train dataset and a test dataset collected in other sessions. As a result, the average cross-session recognition accuracy of 78.7% was obtained on nine subjects, with a sliding window length of 1 s, a step length of 0.4 s, and the six windows. Experimental results showed the proposed MTF-CSP outperforming the compared machine learning and CSP-based methods using the original signals or other features such as time-frequency picture features in terms of accuracy. Further, it is shown that the performance of the AS algorithm is significantly better than that of the Max Voting algorithm adopted in other studies.

Brain-computer interfaces (BCI) based motor imagery (MI) has become a research hotspot for establishing a flexible communication channel for patients with apoplexy or degenerative pathologies. Accurate decoding of motor imagery electroencephalography (MI-EEG) signals, while essential for effective BCI systems, is still challenging due to the significant noise inherent in the EEG signals and the lack of informative correlation between the signals and brain activities. The application of deep learning for EEG feature representation has been rarely investigated, nevertheless bringing improvements to the performance of motor imagery classification. This paper proposes a deep learning decoding method based on multi-hierarchical representation fusion (MHRF) on MI-EEG. It consists of a concurrent framework constructed of bidirectional LSTM (Bi-LSTM) and convolutional neural network (CNN) to fully capture the contextual correlations of MI-EEG and the spectral feature. Also, the stacked sparse autoencoder (SSAE) is employed to concentrate these two domain features into a high-level representation for cross-session and subject training guidance. The experimental analysis demonstrated the efficacy and practicality of the proposed approach using a public dataset from BCI competition IV and a private one collected by our MI task. The proposed approach can serve as a robust and competitive method to improve inter-session and inter-subject transferability, adding anticipation and prospective thoughts to the practical implementation of a calibration-free BCI system.