Modeling Cognitive Load as a Self-Supervised Brain Rate with Electroencephalography and Deep Learning

Abstract

The principal reason for measuring mental workload is to quantify the cognitive cost of performing tasks to predict human performance. Unfortunately, a method for assessing mental workload that has general applicability does not exist yet. This is due to the abundance of intuitions and several operational definitions from various fields that disagree about the sources or workload, its attributes, the mechanisms to aggregate these into a general model and their impact on human performance. This research built upon these issues and presents a novel method for mental workload modelling from EEG data employing deep learning. This method is self-supervised, employing a continuous brain rate, an index of cognitive activation, and does not require human declarative knowledge. The aim is to induce models automatically from data, supporting replicability, generalisability and applicability across fields and contexts. This specific method is a convolutional recurrent neural network trainable with spatially preserving spectral topographic head-maps from EEG data, aimed at fitting a novel brain rate variable. Findings demonstrate the capacity of the convolutional layers to learn meaningful high-level representations from EEG data since within-subject models had, on average, a test Mean Absolute Percentage Error of around 11%. The addition of a Long-Short Term Memory layer for handling sequences of high-level representations was not significant, although it did improve their accuracy. These findings point to the existence of quasi-stable blocks of automatically learnt high-level representations of cognitive activation because they can be induced through convolution and seem not to be dependent on each other over time, intuitively matching the non-stationary nature of brain responses. Additionally, across-subject models, induced with data from an increasing number of participants, thus trained with data containing more variability, obtained a similar accuracy to the within-subject models. This highlights the potential generalisability of the induced high-level representations across people, suggesting the existence of subject-independent cognitive activation patterns. This research contributes to the body of knowledge by providing scholars with a novel computational method for mental workload modelling that aims to be generally applicable and does not rely on ad hoc human crafted models.

Reliability of Mental Workload Index Assessed by EEG with Different Electrode Configurations and Signal Pre-Processing Pipelines

“…While the test–rest approach is valid to prove the stability of results, especially in longitudinal studies, it is not the most suitable test to assess the impact of pre-processing on quantitative estimation when repeated measurements are not provided. In this context, a series of papers have been recently published in which the performances of machine learning approaches to classify the MWL level after different signal pre-processing pipelines were compared [ 36 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 ]. These works are focused only on the automatic classification accuracy, considering several features extracted from all the EEG frequency bands and electrode signals, e.g., ERP, as input to the algorithm, whereas any direct evaluation of the EEG features extracted is provided.…”

Schizo-Net: A novel Schizophrenia Diagnosis Framework Using Late Fusion Multimodal Deep Learning on Electroencephalogram-Based Brain Connectivity Indices

“…In recent years, deep learning methods have been developed to detect various neurological disorders that utilize EEG signals [15], [75] and solve various classification tasks [76], [77], [78]. Although these methods work well in finding hidden features and patterns from the nonlinear data, they struggle to attain higher-classification accuracy on EEG due to the data being highly complex and the frequent non-cerebral contamination that accompanies it [61], [62]. Cortically generated EEG is often contaminated by non-cerebral artifact origins such as eye blinks, ocular movements, Electrocardiogram (ECG), and Electromyogram (EMG) artifacts.…”

Prediction of Cognitive Load from Electroencephalography Signals Using Long Short-Term Memory Network

“…A study investigating prefrontal cortex (PFC) hemodynamics using functional near-infrared spectroscopy (fNIRS) while performing n-back and random number generation (RNG) tasks with multiple cognitive loads suggested a relationship between subjective workload and brain activity [ 7 ]. In attempting to quantify the cognitive load, cognitive load modeling techniques using deep learning are also being studied, considering workload mechanisms and their impact on human performance [ 8 ].…”