Brain Computer Interface for Neuro-rehabilitation With Deep Learning Classification and Virtual Reality Feedback

Abstract

Though Motor Imagery (MI) stroke rehabilitation effectively promotes neural reorganization, current therapeutic methods are immeasurable and their repetitiveness can be demotivating. In this work, a real-time electroencephalogram (EEG) based MI-BCI (Brain Computer Interface) system with a virtual reality (VR) game as a motivational feedback has been developed for stroke rehabilitation. If the subject successfully hits one of the targets, it explodes and thus providing feedback on a successfully imagined and virtually executed movement of hands or feet. Novel classification algorithms with deep learning (DL) and convolutional neural network (CNN) architecture with a unique trial onset detection technique was used. Our classifiers performed better than the previous architectures on datasets from PhysioNet offline database. It provided fine classification in the real-time game setting using a 0.5 second 16 channel input for the CNN architectures. Ten participants reported the training to be interesting, fun and immersive. "It is a bit weird, because it feels like it would be my hands", was one of the comments from a test person. The VR system induced a slight discomfort and a moderate effort for MI activations was reported. We conclude that MI-BCI-VR systems with classifiers based on DL for real-time game applications should be considered for motivating MI stroke rehabilitation. CCS CONCEPTS • Human-centered computing → Empirical studies in HCI; Virtual reality; Usability testing.

A Simplified CNN Classification Method for MI-EEG via the Electrode Pairs Signals

“…In the same method, compared with Dose et al ( 2018 ) and Karácsony et al ( 2019 ), our work achieved superior performance, and used two electrodes to greatly reduce the sample size and data dimensions.…”

“…We can observe that CNN method is indeed effective in MI classification, which can greatly improve the classification accuracy. In the same method, compared with Dose et al (2018) and Karácsony et al (2019), our work achieved superior performance, and used two electrodes to greatly reduce the sample size and data dimensions.…”

“…In general, higher accuracy can be achieved with more electrodes based on the comparison results of Yang et al ( 2015 ) and Cecotti and Graser ( 2010 ). Karácsony et al ( 2019 ) further explained that increasing the number of electrodes can improve the accuracy of classification and recognition without changing the data set and classification method. However, the increase of the number of electrodes will increase the complexity of BCI systems.…”

Flexible Virtual Reality System for Neurorehabilitation and Quality of Life Improvement

“…VR rehabilitation game examples include the use of Microsoft Kinect and Myo gesture control armband (with EMG integrated) for treating people with neurological disorders [ 53 ]. Regarding EEG monitoring, this helps studying the effects of VR training over neural reorganization and can be used as a facilitator for a brain–computer interface (BCI) [ 54 ]. Studies show that BCI leads to better performances in neurorehabilitation systems if it is part of an immersive system [ 55 ].…”

Single-Option P300-BCI Performance Is Affected by Visual Stimulation Conditions

“…Since then, this technology has evolved considerably and nowadays the main applications are found in clinical environments. Areas, such as neuro-rehabilitation for patients with neurodegenerative diseases [ 2 , 3 , 4 , 5 , 6 ], as well as assistive technologies for people with motor impairments [ 7 , 8 , 9 , 10 ], have had the widest presence. However, BCI’s applications have now transcended the clinical environment [ 11 , 12 ].…”