Decoding Hand Motor Imagery Tasks Within the Same Limb From EEG Signals Using Deep Learning

Abstract

Motor imagery (MI) tasks of different body parts have been successfully decoded by conventional classifiers, such as LDA and SVM. On the other hand, decoding MI tasks within the same limb is a challenging problem with these classifiers; however, it would provide more options to control robotic devices. This work proposes to improve the hand MI tasks decoding within the same limb in a brain-computer interface using convolutional neural networks (CNNs); the CNN EEGNet, LDA, and SVM classifiers were evaluated for two (flexion/extension) and three (flexion/extension/grasping) MI tasks. Our approach is the first attempt to apply CNNs for solving this problem to our best knowledge. In addition, visual and electrotactile stimulation were included as BCI training reinforcement after the MI task similar to feedback sessions; then, they were compared. The EEGNet achieved maximum mean accuracies of 78.46% (±12.50%) and 76.72% (±11.67%) for two and three classes, respectively. Outperforming conventional classifiers with results around 60% and 48%, and similar works with results lower than 67% and 75%, respectively. Moreover, the electrical stimulation over the visual stimulus was not significant during the calibration session. The deep learning scheme enhanced the decoding of MI tasks within the same limb against the conventional framework.

Electrotactile feedback applications for hand and arm interactions: A systematic review, meta-analysis, and future directions

“…Finally, as mentioned above, the electrotactile was used with EMG in closed-loop systems. Alternatively, the study of Achanccaray and Hayashibe, (2020) integrated electrotactile feedback with BCI, where an improved motor imagery, and significant improvements in flexion and extension were observed [66]. Given that flexion and extension are the main movements in grasping, the results of this study indicate that electrotactile with BCI systems may show benefits comparable to closed-loop systems.…”

“…4) for stimulating the upper arm, they also observed that the individuals with amputations showed significant improvements in differentiating materials with diverse level of stiffness [79]. Of note, in contrast with the above mentioned studies, which implemented a closed-loop between EMG and electrotactile feedback, the study of Achanccaray and Hayashibe, (2020) implemented the electrotactile feedback in combination with a brain-computer-interface (BCI), where the latter was controlling the prosthetic's movements and actions [66]. Comparably to the closed-loop systems, their study revealed that the conjunction of BCI and electrotactile feedback facilitates a substantially improved motor imagery, and significant improvements in flexion and extension, which are the essential grasping movements [66].…”

Motion prediction for the sensorimotor control of hand prostheses with a brain-machine interface using EEG

“…The Common Spatial patterns technique is probably the most commonly used spatial filtering technique in BMIs [34][35][36]. This method is applied mainly to binary classification; it consists in computing a transformation matrix W that is maximizing the variance of the signal for one class while minimizing it for the other [37].…”

Visual‐Electrotactile Stimulation Feedback to Improve Immersive Brain‐Computer Interface Based on Hand Motor Imagery

“…e CSP method is based on the calculation of a transformation matrix W (equation ( 1)) that maximizes the variance of spatially filtered EEG data belonging to one class while minimizing it for the other class: in our case, EEG data of MI tasks of the left and right limbs. e matrix X is transformed into a matrix Z. W is a square matrix, the dimensions of which depend on the number of channels, and its columns are [11,[41][42][43].…”

“…e SVM classifier has demonstrated to be efficient for discriminating between two motor-imagery classes, and it is the standard classification method used for binary-class BCIs based on MI owing to its fast and computationally efficient training [43][44][45]. An SVM classifier was trained to discriminate between the left and right limbs for the grasping and flexion/extension MI tasks.…”