Cross-Subject Zero Calibration Driver’s Drowsiness Detection: Exploring Spatiotemporal Image Encoding of EEG Signals for Convolutional Neural Network Classification

Paulo, João; Pires, Gabriel; Nunes, Urbano

doi:10.1109/tnsre.2021.3079505

Cited by 52 publications

(11 citation statements)

References 57 publications

(77 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, from our results, deep learning outperformed FBCSP in 4-class MI classification with an improvement of about 25%, but further studies are needed to allow a wider use of this model in real-world scenarios. The use of other input representations can also be tested in the future, e.g., spatio-temporal image encoding representations such as recurrence plots or gramian angular fields successfully used in the context of EEG rhythms classification for drivers drowsiness detection [24]. Finally, DynamicNet showed to be an effective tool for the quick implementation of simple deep learning models, even though it is still in a preliminary version.…”

Section: Discussionmentioning

confidence: 99%

CNN-based Approaches For Cross-Subject Classification in Motor Imagery: From the State-of-The-Art to DynamicNet

Zancanaro

Cisotto

Paulo

et al. 2021

2021 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB)

Self Cite

View full text Add to dashboard Cite

The accurate detection of motor imagery (MI) from electroencephalography (EEG) is a fundamental, as well as challenging, task to provide reliable control of robotic devices to support people suffering from neuro-motor impairments, e.g., in brain-computer interface (BCI) applications. Recently, deep learning approaches have been able to extract subjectindependent features from EEG, to cope with its poor SNR and high intra-subject and cross-subject variability. In this paper, we first present a review of the most recent studies using deep learning for MI classification, with particular attention to their crosssubject performance. Second, we propose DynamicNet, a Pythonbased tool for quick and flexible implementations of deep learning models based on convolutional neural networks. We showcase the potentiality of DynamicNet by implementing EEGNet, a wellestablished architecture for effective EEG classification. Finally, we compare its performance with the filter bank common spatial pattern (FBCSP) in a 4-class MI task (data from a public dataset). To infer cross-subject classification performance, we applied three different cross-validation schemes. From our results, we show that EEGNet implemented with DynamicNet outperforms FBCSP by about 25%, with a statistically significant difference when cross-subject validation schemes are applied. We conclude that deep learning approaches might be particularly helpful to provide higher cross-subject classification performance in multiclass MI classification scenarios. In the future, it is expected to improve DynamicNet to implement new architectures to further investigate cross-subject classification of MI tasks in real-world scenarios.

show abstract

Section: Discussionmentioning

confidence: 99%

CNN-based Approaches For Cross-Subject Classification in Motor Imagery: From the State-of-The-Art to DynamicNet

Zancanaro

Cisotto

Paulo

et al. 2021

2021 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The first two items in the table used CNN for classification, the next three items applied domain adaptation to classification, and the last one improved easy transfer learning (EasyTL) as the classifier. Paulo et al [55] applied a Butterworth filter to get the power density of the theta (4-8 Hz), alpha (8-13 Hz), and beta (13-30 Hz). Then they calculated attention metrics associated with fatigue.…”

Section: Eeg-based Fatigue Detection In Cross-subjectmentioning

confidence: 99%

Recent Advances in Fatigue Detection Algorithm Based on EEG

Wang¹,

Wan²,

Li³

et al. 2023

Intelligent Automation &Amp; Soft Computing

View full text Add to dashboard Cite

Fatigue is a state commonly caused by overworked, which seriously affects daily work and life. How to detect mental fatigue has always been a hot spot for researchers to explore. Electroencephalogram (EEG) is considered one of the most accurate and objective indicators. This article investigated the development of classification algorithms applied in EEG-based fatigue detection in recent years. According to the different source of the data, we can divide these classification algorithms into two categories, intra-subject (within the same subject) and cross-subject (across different subjects). In most studies, traditional machine learning algorithms with artificial feature extraction methods were commonly used for fatigue detection as intra-subject algorithms. Besides, deep learning algorithms have been applied to fatigue detection and could achieve effective result based on large-scale dataset. However, it is difficult to perform long-term calibration training on the subjects in practical applications. With the lack of large samples, transfer learning algorithms as a cross-subject algorithm could promote the practical application of fatigue detection methods. We found that the research based on deep learning and transfer learning has gradually increased in recent years. But as a field with increasing requirements, researchers still need to continue to explore efficient decoding algorithms, design effective experimental paradigms, and collect and accumulate valid standard data, to achieve fast and accurate fatigue detection methods or systems to further widely apply.

show abstract

“…Unlike this reshaped signal operation, some sequence-toimage methods, such as GAF [14], MTF [14], and recurrence plot (RP) [23], can encode one-dimensional signals into twodimensional images. GAF and RP encode EEG signals into image representations for drowsiness detection [24]. Xiao et al [25] extracted features from ECG images generated by GAF to classify hand movements.…”

Section: B Encoding Time Series To Imagesmentioning

confidence: 99%

A General and Scalable Vision Framework for Functional Near-Infrared Spectroscopy Classification

Wang

Zhang

Xia

et al. 2022

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Functional near-infrared spectroscopy (fNIRS), a non-invasive optical technique, is widely used to monitor brain activities for disease diagnosis and braincomputer interfaces (BCIs). Deep learning-based fNIRS classification faces three major barriers: limited datasets, confusing evaluation criteria, and domain barriers. We apply more appropriate evaluation methods to three openaccess datasets to solve the first two barriers. For domain barriers, we propose a general and scalable vision fNIRS framework that converts multi-channel fNIRS signals into multi-channel virtual images using the Gramian angular difference field (GADF). We use the framework to train state-of-the-art visual models from computer vision (CV) within a few minutes, and the classification performance is competitive with the latest fNIRS models. In crossvalidation experiments, visual models achieve the highest average classification results of 78.68% and 73.92% on mental arithmetic and word generation tasks, respectively. Although visual models are slightly lower than the fNIRS models on unilateral finger-and foot-tapping tasks, the F1score and kappa coefficient indicate that these differences are insignificant in subject-independent experiments. Furthermore, we study fNIRS signal representations and the classification performance of sequence-to-image methods. We hope to introduce rich achievements from the CV domain to improve fNIRS classification research.

show abstract

Cross-Subject Zero Calibration Driver’s Drowsiness Detection: Exploring Spatiotemporal Image Encoding of EEG Signals for Convolutional Neural Network Classification

Cited by 52 publications

References 57 publications

CNN-based Approaches For Cross-Subject Classification in Motor Imagery: From the State-of-The-Art to DynamicNet

CNN-based Approaches For Cross-Subject Classification in Motor Imagery: From the State-of-The-Art to DynamicNet

Recent Advances in Fatigue Detection Algorithm Based on EEG

A General and Scalable Vision Framework for Functional Near-Infrared Spectroscopy Classification

Contact Info

Product

Resources

About