Compact convolutional neural networks for classification of asynchronous steady-state visual evoked potentials

Abstract

Objective. Steady-State Visual Evoked Potentials (SSVEPs) are neural oscillations from the parietal and occipital regions of the brain that are evoked from flickering visual stimuli. SSVEPs are robust signals measurable in the electroencephalogram (EEG) and are commonly used in brain-computer interfaces (BCIs). However, methods for high-accuracy decoding of SSVEPs usually require hand-crafted approaches that leverage domain-specific knowledge of the stimulus signals, such as specific temporal frequencies in the visual stimuli and their relative spatial arrangement. When this knowledge is unavailable, such as when SSVEP signals are acquired asynchronously, such approaches tend to fail. Approach. In this paper, we show how a compact convolutional neural network (Compact-CNN), which only requires raw EEG signals for automatic feature extraction, can be used to decode signals from a 12-class SSVEP dataset without the need for user-specific calibration. Main results. The Compact-CNN demonstrates across subject mean accuracy of approximately 80 %, out-performing current state-of-the-art, handcrafted approaches using canonical correlation analysis (CCA) and Combined-CCA. Furthermore, the Compact-CNN approach can reveal the underlying feature representation, revealing that the deep learner extracts additional phase-and amplitude-related features associated with the structure of the dataset. Significance. We discuss how our Compact-CNN shows promise for BCI applications that allow users to freely gaze/attend to any stimulus at any time (e.g., asynchronous BCI) as well as provides a method for analyzing SSVEP signals in a way that might augment our understanding about the basic processing in the visual cortex.Evoked potentials are robust signals in the electroencephalogram (EEG) induced by sensory stimuli, and they have been used to study normal and abnormal function of the sensory cortex [1]. The most well-studied of these are Steady-State Visual Evoked Potentials (SSVEPs), which are neural oscillations in the visual cortex that are evoked from stimuli that temporally flicker in a narrow frequency band [2,3]. SSVEPs likely arise from a reorganization of spontaneous intrinsic brain oscillations in response to a stimulus [4]. Paradigms leveraging SSVEP responses have been used to investigate the organization of the visual system [5,6], identify biomarkers of disease and sensory function [7][8][9], and probe visual perception [10,11].The robustness of SSVEP has enabled its use as a control signal for brain computer interfaces (BCIs) that enable low-bandwith communication for individuals with catastrophic loss of motor functions, bypassing neuro-muscular pathways and establishing a communication link directly to the brain [12,13]. In a typical SSVEP BCI, a patient/subject is presented with a grid of squares on a computer monitor, where each square contains semantic information such as a letter, number, character, or action. Superimposed on these squares are visual flicker frequencies that uniquely "tag" each square, thus mapp...

Deep learning-based electroencephalography analysis: a systematic review

“…For deeper layers, however, the hierarchical nature of neural networks means it is much harder to understand what a weight is applied to. The analysis of model activations was used in multiple studies [212,194,87,83,208,167,154,109]. This kind of inspection method usually involves visualizing the activations of the trained model over multiple examples, and thus inferring how different parts of the network react to known inputs.…”

“…Occlusion sensitivity techniques [92,26,175] use a similar idea, by which the decisions of the network when different parts of the input are occluded are analyzed. [135,211,86,34,87,200,182,122,170,228,164,109,204,85,25] Analysis of activations [212,194,87,83,208,167,154,109] Input-perturbation network-prediction correlation maps [149,191,67,16,150] Generating input to maximize activation [188,144,160,15] Occlusion of input [92,26,175] Several studies used backpropagation-based techniques to generate input maps that maximize activations of specific units [188,144,160,15]. These maps can then be used to infer the role of specific neurons, or the kind of input they are sensitive to.…”

Universal Joint Feature Extraction for P300 EEG Classification Using Multi-Task Autoencoder

“…In Experiment B, EEGNet and DeepConvNet were pretrained in the same way as ERPENet. Regardless of the compact size in EEGnet and the claim in [28] that EEGNet could be trained with very limited data, EEGNet did not perform well in BCI-COMP which is the smallest dataset in this evaluation. Table 6 also shows that ERPENet outperforms DeepConvNet, while DeepConvNet has comparable results as in EEGNet, which is consistent with the experiments with P300 dataset in [28].…”

“…Then, the ERPENet was adopted as a pre-trained network to the attended and unattended event classification network. The results are compared to the state-of-the-art P300 dimensionality reduction algorithm [26] named Xdawn with Bayesian LDA classification [27] and state-of-the-art in EEG classification deep learning models, EEGNet [28] and DeepConvNet [29]. EEGNet and DeepConvNet were deep learning models, designed for various EEG classification tasks, and yielded state-of-the-art results.…”

Decoding P300 Variability using Convolutional Neural Networks

“…Recent work by [13] has yielded a CNN architecture for EEG (EEGNet) that can learn from relatively small amounts of data, on the order of hundreds of trials per subject. Furthermore, [13,44] showed that EEGNet enabled cross-subject transfer performance equal to or better than conventional approaches for several EEG classification paradigms, both event-related and oscillatory. EEGNet is also the model used to obtain our crossexperiment results described in [26,39,45].…”