Classification of Motor Imagery EEG Signals with multi-input Convolutional Neural Network by augmenting STFT

Shovon, Tanvir Hasan; Nazi, Zabir Al; Dash, Shovon; Hossain, Md. Foisal

doi:10.1109/icaee48663.2019.8975578

Cited by 46 publications

(23 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shovon et al (2019) applied STFT on EEG signals to transform signal to images for binary classification of motor-imagery signals [114]. They used rotation, flipping, zoom in and zoom out as DA techniques to overcome the overfitting problem in their proposed CNN model.…”

Section: Othermentioning

confidence: 99%

Data augmentation for deep-learning-based electroencephalography

Lashgari

Liang

Maoz

2020

Journal of Neuroscience Methods

277

140

View full text Add to dashboard Cite

Highlights • Data augmentation (DA) is increasingly used with deep learning (DL) on EEG • It enhances decoding accuracy left unexplained by 29% on average on the datasets we review • We analyze which specific DA techniques appear to work best for which EEG tasks • We tested various DA techniques on an open motor-imagery task and compared the accuracy gains to demonstrate the usefulness of DA for DL-based EEG analysis • We propose guidelines for reporting parameters for different DA techniques Abstract-Background Data augmentation (DA) has recently been demonstrated to achieve considerable performance gains for deep learning (DL)-increased accuracy and stability and reduced overfitting. Some electroencephalography (EEG) tasks suffer from low samples-to-features ratio, severely reducing DL effectiveness. DA with DL thus holds transformative promise for EEG processing, possibly like DL revolutionized computer vision, etc.-New method We review trends and approaches to DA for DL in EEG to address: Which DA approaches exist and are common for which EEG tasks? What input features are used? And, what kind of accuracy gain can be expected?-Results DA for DL on EEG begun 5 years ago and is steadily used more. We grouped DA techniques (noise addition, generative adversarial networks, sliding windows, sampling, Fourier transform, recombination of segmentation, and others) and EEG tasks (into seizure detection, sleep stages, motor imagery, mental workload, emotion recognition, motor tasks, and visual tasks). DA efficacy across techniques varied considerably. Noise addition and sliding windows provided the highest accuracy boost; mental workload most benefitted from DA. Sliding window, noise addition, and sampling methods most common for seizure detection, mental workload, and sleep stages, respectively.-Comparing with existing methods Percent of decoding accuracy explained by DA beyond unaugmented accuracy varied between 8% for recombination of segmentation and 36% for noise addition and from 14% for motor imagery to 56% for mental workload-29% on average.-Conclusions DA increasingly used and considerably improved DL decoding accuracy on EEG. Additional publications-if adhering to our reporting guidelines-will facilitate more detailed analysis.

show abstract

Section: Othermentioning

confidence: 99%

Data augmentation for deep-learning-based electroencephalography

Lashgari

Liang

Maoz

2020

Journal of Neuroscience Methods

277

140

View full text Add to dashboard Cite

show abstract

“…Motor imagery [32] Geometric transformation Easy to lose motion-related information after the geometric transformation…”

Section: Electroencephalogram (Eeg) Pattern Augmentation Methods Limimentioning

confidence: 99%

Data Augmentation for Motor Imagery Signal Classification Based on a Hybrid Neural Network

Zhang

Han

et al. 2020

Sensors

View full text Add to dashboard Cite

As an important paradigm of spontaneous brain-computer interfaces (BCIs), motor imagery (MI) has been widely used in the fields of neurological rehabilitation and robot control. Recently, researchers have proposed various methods for feature extraction and classification based on MI signals. The decoding model based on deep neural networks (DNNs) has attracted significant attention in the field of MI signal processing. Due to the strict requirements for subjects and experimental environments, it is difficult to collect large-scale and high-quality electroencephalogram (EEG) data. However, the performance of a deep learning model depends directly on the size of the datasets. Therefore, the decoding of MI-EEG signals based on a DNN has proven highly challenging in practice. Based on this, we investigated the performance of different data augmentation (DA) methods for the classification of MI data using a DNN. First, we transformed the time series signals into spectrogram images using a short-time Fourier transform (STFT). Then, we evaluated and compared the performance of different DA methods for this spectrogram data. Next, we developed a convolutional neural network (CNN) to classify the MI signals and compared the classification performance of after DA. The Fréchet inception distance (FID) was used to evaluate the quality of the generated data (GD) and the classification accuracy, and mean kappa values were used to explore the best CNN-DA method. In addition, analysis of variance (ANOVA) and paired t-tests were used to assess the significance of the results. The results showed that the deep convolutional generative adversarial network (DCGAN) provided better augmentation performance than traditional DA methods: geometric transformation (GT), autoencoder (AE), and variational autoencoder (VAE) (p < 0.01). Public datasets of the BCI competition IV (datasets 1 and 2b) were used to verify the classification performance. Improvements in the classification accuracies of 17% and 21% (p < 0.01) were observed after DA for the two datasets. In addition, the hybrid network CNN-DCGAN outperformed the other classification methods, with average kappa values of 0.564 and 0.677 for the two datasets.

show abstract

“…To reduce the high dimension, dimensionality reduction techniques such as Principal Component Analysis (PCA) [4], [12], [18], Independent Component Analysis(ICA) [18] and Linear Discriminative Analysis(LDA) [13], [14], [21] [10]. Recently, deep learning models CNN [26,27] and Long Short Term Memory (LSTM) [28] networks are employed to improve the efficacy of the EEG signal classifier. The convolutional neural network architecture employed in paper [27] achieved an accuracy of 71/82% (+4.2%) with a multichannel EEG signal as 2D image input.…”

Section: Introductionmentioning

confidence: 99%

“…The convolutional neural network architecture employed in paper [27] achieved an accuracy of 71/82% (+4.2%) with a multichannel EEG signal as 2D image input. The accuracy got much improved when the convolution network was employed using the transfer learning technique [26]. The pre-trained model AlexNet is adapted for the classification of EEG signals through transfer learning [29].…”

Section: Introductionmentioning

confidence: 99%

“…The pre-trained model AlexNet is adapted for the classification of EEG signals through transfer learning [29]. Also, ResNet is adapted for MI EEG classification [26]. However, these networks improved the accuracy of the classification at the cost of huge computational complexity Although a lot of research has been carried out to improve the intra-subject accuracy and inter-subject accuracy of the BCI task, accuracy is still less than 80%.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Deep Learning Approach for Robotic Arm Control using Brain-Computer Interface

2020

International Journal of Biology and Biomedical Engineering

View full text Add to dashboard Cite

Brain-Computer Interface (BCI) is atechnology that enables a human to communicate with anexternal stratagem to achieve the desired result. This paperpresents a Motor Imagery (MI) – Electroencephalography(EEG) signal based robotic hand movements of lifting anddropping of an external robotic arm. The MI-EEG signalswere extracted using a 3-channel electrode system with theAD8232 amplifier. The electrodes were placed on threelocations, namely, C3, C4, and right mastoid. Signalprocessing methods namely, Butterworth filter and Sym-9Wavelet Packet Decomposition (WPD) were applied on theextracted EEG signals to de-noise the raw EEG signal.Statistical features like entropy, variance, standarddeviation, covariance, and spectral centroid were extractedfrom the de-noised signals. The statistical features werethen applied to train a Multi-Layer Perceptron (MLP) -Deep Neural Network (DNN) to classify the hand movementinto two classes; ‘No Hand Movement’ and ’HandMovement’. The resultant k-fold cross-validated accuracyachieved was 85.41% and other classification metrics, suchas precision, recall sensitivity, specificity, and F1 Score werealso calculated. The trained model was interfaced withArduino to move the robotic arm according to the classpredicted by the DNN model in a real-time environment.The proposed end to end low-cost deep learning frameworkprovides a substantial improvement in real-time BCI.

show abstract

Classification of Motor Imagery EEG Signals with multi-input Convolutional Neural Network by augmenting STFT

Cited by 46 publications

References 14 publications

Data augmentation for deep-learning-based electroencephalography

Data augmentation for deep-learning-based electroencephalography

Data Augmentation for Motor Imagery Signal Classification Based on a Hybrid Neural Network

A Deep Learning Approach for Robotic Arm Control using Brain-Computer Interface

Contact Info

Product

Resources

About