Generalized Deep Learning EEG Models for Cross-Participant and Cross-Task Detection of the Vigilance Decrement in Sustained Attention Tasks

Abstract

Tasks which require sustained attention over a lengthy period of time have been a focal point of cognitive fatigue research for decades, with these tasks including air traffic control, watchkeeping, baggage inspection, and many others. Recent research into physiological markers of mental fatigue indicate that markers exist which extend across all individuals and all types of vigilance tasks. This suggests that it would be possible to build an EEG model which detects these markers and the subsequent vigilance decrement in any task (i.e., a task-generic model) and in any person (i.e., a cross-participant model). However, thus far, no task-generic EEG cross-participant model has been built or tested. In this research, we explored creation and application of a task-generic EEG cross-participant model for detection of the vigilance decrement in an unseen task and unseen individuals. We utilized three different models to investigate this capability: a multi-layer perceptron neural network (MLPNN) which employed spectral features extracted from the five traditional EEG frequency bands, a temporal convolutional network (TCN), and a TCN autoencoder (TCN-AE), with these two TCN models being time-domain based, i.e., using raw EEG time-series voltage values. The MLPNN and TCN models both achieved accuracy greater than random chance (50%), with the MLPNN performing best with a 7-fold CV balanced accuracy of 64% (95% CI: 0.59, 0.69) and validation accuracies greater than random chance for 9 of the 14 participants. This finding demonstrates that it is possible to classify a vigilance decrement using EEG, even with EEG from an unseen individual and unseen task.

Cross-task-oriented EEG signal analysis methods: Our opinion

“…From the perspective of feature classification, although the differences between tasks corresponding to different EEG signals limit the cross-task versatility of existing classification models (Zhou et al, 2022 ), researchers are still committed to finding or constructing some relatively general cross-task feature classification models (Kamrud et al, 2021 ; Mota et al, 2021 ; Taori et al, 2022 ).…”

“…(1) Neural network method. Kamrud et al studied the commonality of three different models in terms of cross-task: MLPNN, temporal convolutional network (TCN), TCN auto encoder (TCN-AE) (Kamrud et al, 2021 ), and the results showed that the best model for cross-task classification was the MLPNN frequency domain model.…”

“…The objective aimed to find ways to effectively deal with the differences between tasks. At the same time, some researchers have explored the classifiers which are more friendly to the differences between different tasks by comparing with the traditional feature classification methods, including multi-layer perceptron neural network (MLPNN) (Kamrud et al, 2021 ), domain adaptive methods (Zhou et al, 2022 ), sliding-window support vector machine (SVM) (Boring et al, 2020 ), etc. On the other hand, some new cross-task models based on deep learning models were proposed to narrow the differences between tasks, such as convolutional neural networks (CNNs) (Mota et al, 2021 ), recurrent neural networks (RNNs) (Gupta et al, 2021 ), metric-based methods (Jia et al, 2023 ), combinations of CNNs and RNNs (Zhang et al, 2019 ; Zhou et al, 2019 ; Taori et al, 2022 ), etc.…”

“…On the other hand, some new cross-task models based on deep learning models were proposed to narrow the differences between tasks, such as convolutional neural networks (CNNs) (Mota et al, 2021 ), recurrent neural networks (RNNs) (Gupta et al, 2021 ), metric-based methods (Jia et al, 2023 ), combinations of CNNs and RNNs (Zhang et al, 2019 ; Zhou et al, 2019 ; Taori et al, 2022 ), etc. However, there are still many unexplored areas in the field of cross-task EEG signal analysis methods, such as: task segmentation and complexity design (Kamrud et al, 2021 ), multi-source domain adaptive application (Zhou et al, 2022 ), multi-scale and multi-directional filter research (Taori et al, 2022 ), considering both feature extraction and feature classification, and increasing the amount of data. Furthermore, there are also some interconnections between cross-task analysis and relatively common cross-subject studies.…”

Evaluation of a New Lightweight EEG Technology for Translational Applications of Passive Brain-Computer Interfaces

“…A physiological decrease in vigilance over time is associated with performance degradation, such as slower reaction times and loss of situation awareness, whereas optimal performance is ensured by an adequate level of activation throughout the task (Parasuraman et al, 1998 ). EEG measures have been used as features for machine learning models to monitor vigilance levels in different contexts (Sebastiani et al, 2020 ; Kamrud et al, 2021 ; Li and Chung, 2022 ).…”