A Multi-Scale Temporal Convolutional Network with Attention Mechanism for Force Level Classification during Motor Imagery of Unilateral Upper-Limb Movements

Abstract

In motor imagery (MI) brain–computer interface (BCI) research, some researchers have designed MI paradigms of force under a unilateral upper-limb static state. It is difficult to apply these paradigms to the dynamic force interaction process between the robot and the patient in a brain-controlled rehabilitation robot system, which needs to induce thinking states of the patient’s demand for assistance. Therefore, in our research, according to the movement of wiping the table in human daily life, we designed a three-level-force MI paradigm under a unilateral upper-limb dynamic state. Based on the event-related de-synchronization (ERD) feature analysis of the electroencephalography (EEG) signals generated by the brain’s force change motor imagination, we proposed a multi-scale temporal convolutional network with attention mechanism (MSTCN-AM) algorithm to recognize ERD features of MI-EEG signals. Aiming at the slight feature differences of single-trial MI-EEG signals among different levels of force, the MSTCN module was designed to extract fine-grained features of different dimensions in the time–frequency domain. The spatial convolution module was then used to learn the area differences of space domain features. Finally, the attention mechanism dynamically weighted the time–frequency–space domain features to improve the algorithm’s sensitivity. The results showed that the accuracy of the algorithm was 86.4 ± 14.0% for the three-level-force MI-EEG data collected experimentally. Compared with the baseline algorithms (OVR-CSP+SVM (77.6 ± 14.5%), Deep ConvNet (75.3 ± 12.3%), Shallow ConvNet (77.6 ± 11.8%), EEGNet (82.3 ± 13.8%), and SCNN-BiLSTM (69.1 ± 16.8%)), our algorithm had higher classification accuracy with significant differences and better fitting performance.

Characterization and classification of kinesthetic motor imagery levels

“…Secondly, a high multi-classification accuracy of 96.42% has been achieved by using EEG-KMI data from the PM, DLPFC, and near IPL areas. This result supports the hypothesis that KMI-BCI 4-multilevel detection can be achieved without necessarily including EEG data from the M1 area, as in [58,61,64], and is consistent with the results of [23] where it was found high accuracy of binary classification for KMI and VMI using PM, DLPFC, IPL, and SMA; however, they used high-density EEG (64 electrodes). This suggests a novel venue for research, mainly using lowres and low-cost EEG headsets, such as the Emotiv used in this study.…”

“…Similar works that detect KMI levels made their analysis from motor cortex EEG signals, thus is, focusing on the information provided by primary motor cortex (M1) electrodes C3, C4, and even Cz [60][61][62][63][64], their classification accuracy reached 70.9%, 78.3%, 53%-55%, 81.4%, and 86.4%, respectively. Table 4 presents, for comparison purposes, previous similar works that have classified multi-level KMI.…”

“…The processing techniques reported in multilevel KMI studies include the CSP, ERD, and movementrelated cortical potentials for the characterization stage, SVM-recursive feature elimination (SVM-RFE), Linear Discriminant Analysis, temporal convolutional network, SVM, and MP for classification. The reported accuracy in these studies has been 70.9% [60], 78.3% [61], 53%-55% [62], 81.4% [63], and 86.4% [64]. Recently, the implementation of the Hurst algorithm with variogram for EEG characterization of three wrist and forearm KMI tasks has been reported [81].…”

“…In another study, functional near-infrared spectroscopy (fNIRS) was combined with EEG data to increase the classification accuracy of the KMI task to distinguish speed against force (binary classification) [59]. The studies that used KMI levels focused their analysis on data collected mainly from electrodes C3 and C4 over the M1 (the detail of these studies are presented in table 4) [60][61][62][63][64].…”

Event-Related EEG Desynchronization Reveals Enhanced Motor Imagery From the Third Person Perspective by Manipulating Sense of Body Ownership With Virtual Reality for Stroke Patients

“…This approach is a common choice for assessing the effectiveness of MI involving hand movements in healthy [25], [55] and stroke patients [56]. The C3/C4 channels were used to evaluate MI involving more complex upper limb tasks, such as wiping a table [57] and elbow rotation, among others [22]. We also focused our analysis on C3/C4, which are the most representative channels for the primary motor areas and commonly used for the upper limb rehabilitation assessment.…”