Electroencephalogram and surface electromyogram fusion-based precise detection of lower limb voluntary movement using convolution neural network-long short-term memory model

Abstract

The electroencephalogram (EEG) and surface electromyogram (sEMG) fusion has been widely used in the detection of human movement intention for human–robot interaction, but the internal relationship of EEG and sEMG signals is not clear, so their fusion still has some shortcomings. A precise fusion method of EEG and sEMG using the CNN-LSTM model was investigated to detect lower limb voluntary movement in this study. At first, the EEG and sEMG signal processing of each stage was analyzed so that the response time difference between EEG and sEMG can be estimated to detect lower limb voluntary movement, and it can be calculated by the symbolic transfer entropy. Second, the data fusion and feature of EEG and sEMG were both used for obtaining a data matrix of the model, and a hybrid CNN-LSTM model was established for the EEG and sEMG-based decoding model of lower limb voluntary movement so that the estimated value of time difference was about 24 ∼ 26 ms, and the calculated value was between 25 and 45 ms. Finally, the offline experimental results showed that the accuracy of data fusion was significantly higher than feature fusion-based accuracy in 5-fold cross-validation, and the average accuracy of EEG and sEMG data fusion was more than 95%; the improved average accuracy for eliminating the response time difference between EEG and sEMG was about 0.7 ± 0.26% in data fusion. In the meantime, the online average accuracy of data fusion-based CNN-LSTM was more than 87% in all subjects. These results demonstrated that the time difference had an influence on the EEG and sEMG fusion to detect lower limb voluntary movement, and the proposed CNN-LSTM model can achieve high performance. This work provides a stable and reliable basis for human–robot interaction of the lower limb exoskeleton.

Analysis of Cortico-Muscular Coupling and Functional Brain Network under Different Standing Balance Paradigms

“…TE has unique advantages in the analysis of bioelectrical signals. It can be used not only to untangle complex relationships between signals, but also to analyze the functional connection and information transmission of biological systems [13,14].…”

Multi-layer CNN-LSTM network with self-attention mechanism for robust estimation of nonlinear uncertain systems

“…With the assistance of robotic technology, intelligent rehabilitation therapy can be realized to reduce the workload of clinical medical staff and improve the efficiency of patients’ rehabilitation training ( Kapelner et al, 2020 ). In the human–machine interaction between rehabilitation robots and patients, traditional human–machine interaction techniques often involve the robot passively receiving instructions, which may not be convenient for patients with motor function impairments ( Zhai et al, 2017 ; Zhang et al, 2022 ). In recent years, human–machine interaction technology needs to evolve toward allowing robots to actively understand human behavioral intentions, resulting in a new type of interaction based on human biological signals.…”