Fusion of Spatial, Temporal, and Spectral EEG Signatures Improves Multilevel Cognitive Load Prediction

Liu, Yingxin; Yu, Yang; Ye, Zeqi; Li, Ming; Zhang, Yifan; Zhou, Zongtan; Hu, Dewen; Zeng, Ling‐Li

doi:10.1109/thms.2023.3235003

Cited by 20 publications

(5 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research confirms that subjective evaluation of MWL correlates well with TL [19,[22][23][24][25][26][27][28]. This is true when TL is modified through changes in subtasks complexities (qualitatively) [19,[23][24][25], as well as through variations in the number of subtasks (quantitatively) [26][27][28].…”

Section: Related Worksupporting

confidence: 62%

“…This metric demonstrates correlation with objective TL measures across various range of tasks [13][14][15]. Other band powers like beta (13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30) [16], ratio of beta and alpha power [17], as well as various other power ratios across different bands are also used as indicators of MWL [18][19][20][21].…”

Section: Related Workmentioning

confidence: 99%

“…This is particularly true when the number of occurrences of subtasks an operator handles is the predominant contributor of perceived MWL. However, the progression in machine learning [22], and particularly the rise of deep learning applied in the EEG field [32][33][34][35][36][37][38], is opening new possibilities for discovering relations between MWL and EEG signals. These approaches can bypass the intermediary steps of feature-engineering, leveraging the possibility of fully automatic end-to-end learning.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study

Pušica,

Kartali,

Bojović

et al. 2024

Brain Sciences

View full text Add to dashboard Cite

While the term task load (TL) refers to external task demands, the amount of work, or the number of tasks to be performed, mental workload (MWL) refers to the individual’s effort, mental capacity, or cognitive resources utilized while performing a task. MWL in multitasking scenarios is often closely linked with the quantity of tasks a person is handling within a given timeframe. In this study, we challenge this hypothesis from the perspective of electroencephalography (EEG) using a deep learning approach. We conducted an EEG experiment with 50 participants performing NASA Multi-Attribute Task Battery II (MATB-II) under 4 different task load levels. We designed a convolutional neural network (CNN) to help with two distinct classification tasks. In one setting, the CNN was used to classify EEG segments based on their task load level. In another setting, the same CNN architecture was trained again to detect the presence of individual MATB-II subtasks. Results show that, while the model successfully learns to detect whether a particular subtask is active in a given segment (i.e., to differentiate between different subtasks-related EEG patterns), it struggles to differentiate between the two highest levels of task load (i.e., to distinguish MWL-related EEG patterns). We speculate that the challenge comes from two factors: first, the experiment was designed in a way that these two highest levels differed only in the quantity of work within a given timeframe; and second, the participants’ effective adaptation to increased task demands, as evidenced by low error rates. Consequently, this indicates that under such conditions in multitasking, EEG may not reflect distinct enough patterns to differentiate higher levels of task load.

show abstract

Section: Related Worksupporting

confidence: 62%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study

Pušica,

Kartali,

Bojović

et al. 2024

Brain Sciences

View full text Add to dashboard Cite

show abstract

“…Second, artefacts were automatically removed via the Automatic Artefact Removal (AAR) plug-in for EEGLAB. The AAR method is suitable for real-time applications [44], [45]. Third, all trials with reaction times of less than 100 ms were removed.…”

Section: B Data Processingmentioning

confidence: 99%

A Connectivity-Aware Graph Neural Network for Real-Time Drowsiness Classification

Zhuang,

Wang,

Chang

et al. 2024

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Drowsy driving is one of the primary causes of driving fatalities. Electroencephalography (EEG), a method for detecting drowsiness directly from brain activity, has been widely used for detecting driver drowsiness in real-time. Recent studies have revealed the great potential of using brain connectivity graphs constructed based on EEG data for drowsy state predictions. However, traditional brain connectivity networks are irrelevant to the downstream prediction tasks. This article proposes a connectivity-aware graph neural network (CAGNN) using a selfattention mechanism that can generate task-relevant connectivity networks via end-to-end training. Our method achieved an accuracy of 72.6% and outperformed other convolutional neural networks (CNNs) and graph generation methods based on a drowsy driving dataset. In addition, we introduced a squeeze-andexcitation (SE) block to capture important features and demonstrated that the SE attention score can reveal the most important feature band. We compared our generated connectivity graphs in the drowsy and alert states and found drowsiness connectivity patterns, including significantly reduced occipital connectivity and interregional connectivity. Additionally, we performed a post hoc interpretability analysis and found that our method could identify drowsiness features such as alpha spindles. Our code is available online 1 .

show abstract

“…In previous studies, univariate features such as the coefficients of auto-regression (AR) (Campisi and Rocca, 2014;Rocca et al, 2014;Keshishzadeh et al, 2016;Bhateja et al, 2019), power spectral density (PSD) (Palaniappan and Mandic, 2007;Ashby et al, 2011;Pham et al, 2015;Di et al, 2019), and wavelet transform (WT) (Kumari and Vaish, 2015;Kaur et al, 2017;Alyasseri et al, 2018;Yang et al, 2018) have been widely used to represent individual differences in EEG. As they are obtained by calculating signals from each electrode, univariate features are sensitive to changes in EEG amplitudes (Liu et al, 2023), which may amplify the intra-person variation. Bivariate features such as the connectivity between channels can effectively be less sensitive to amplitude interference owing to inevitably physiological or psychological factors (Zhang et al, 2021(Zhang et al, , 2022.…”

Section: Introductionmentioning

confidence: 99%

Fusing the spatial structure of electroencephalogram channels can increase the individualization of the functional connectivity network

Li,

Yang,

Tian

et al. 2023

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

An electroencephalogram (EEG) functional connectivity (FC) network is individualized and plays a significant role in EEG-based person identification. Traditional FC networks are constructed by statistical dependence and correlation between EEG channels, without considering the spatial relationships between the channels. The individual identification algorithm based on traditional FC networks is sensitive to the integrity of channels and crucially relies on signal preprocessing; therefore, finding a new presentation for FC networks may help increase the performance of the identification algorithms. EEG signals are smooth across space owing to the volume conduction effect. Considering such spatial relationships among channels can provide a more accurate representation of FC networks. In this study, we propose an EEG FC network with virtual nodes that combines the spatial relationships and functional connectivity of channels. The comparison results for individual identification show that the novel EEG network is more individualized and achieves an accuracy of 98.64% for data without preprocessing. Furthermore, our algorithm is more robust in reducing the number of channels and can perform well even when a large area of channels is removed.

show abstract

Fusion of Spatial, Temporal, and Spectral EEG Signatures Improves Multilevel Cognitive Load Prediction

Cited by 20 publications

References 47 publications

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study

A Connectivity-Aware Graph Neural Network for Real-Time Drowsiness Classification

Fusing the spatial structure of electroencephalogram channels can increase the individualization of the functional connectivity network

Contact Info

Product

Resources

About