Datasets for Cognitive Load Inference Using Wearable Sensors and Psychological Traits

Abstract

This study introduces two datasets for multimodal research on cognitive load inference and personality traits. Different to other datasets in Affective Computing, which disregard participants’ personality traits or focus only on emotions, stress, or cognitive load from one specific task, the participants in our experiments performed seven different tasks in total. In the first dataset, 23 participants played a varying difficulty (easy, medium, and hard) game on a smartphone. In the second dataset, 23 participants performed six psychological tasks on a PC, again with varying difficulty. In both experiments, the participants filled personality trait questionnaires and marked their perceived cognitive load using NASA-TLX after each task. Additionally, the participants’ physiological response was recorded using a wrist device measuring heart rate, beat-to-beat intervals, galvanic skin response, skin temperature, and three-axis acceleration. The datasets allow multimodal study of physiological responses of individuals in relation to their personality and cognitive load. Various analyses of relationships between personality traits, subjective cognitive load (i.e., NASA-TLX), and objective cognitive load (i.e., task difficulty) are presented. Additionally, baseline machine learning models for recognizing task difficulty are presented, including a multitask learning (MTL) neural network that outperforms single-task neural network by simultaneously learning from the two datasets. The datasets are publicly available to advance the field of cognitive load inference using commercially available devices.

Ultra-Short Window Length and Feature Importance Analysis for Cognitive Load Detection from Wearable Sensors

“…The CogLoad dataset from [14] was used in this study. The dataset includes 23 participants (7 females, mean age 29.5 years with a standard deviation of 10.1 years) who solved cognitive tasks of varying difficulty.…”

“…After each task, the participants were asked to fill in the NASA-TLX questionnaire to determine their subjective cognitive load, however, those questionnaires were not utilized here. Further details on the study protocol and tasks can be found in [14].…”

“…The changes in the cognitive load are also reflected in various biosignals that can be measured by using biosensors, e.g., wearable devices [13,14]. For instance, increasing task difficulty (or cognitive load) and acute stress increases the HR and breathing rate [15], ST [11] as well as GSR [16] and decreases the HRV [17], number of eye movements [18], and increases the blink rate [19,20].…”

Can We Ditch Feature Engineering? End-to-End Deep Learning for Affect Recognition from Physiological Sensor Data

“…One reason for such behavior may be the fact that physiological responses induced by the emotional videos used in the emotion datasets are generally less significant than responses invoked by the Trier Social Stress Test used in WESAD. The difficulty of recognizing subtle affective stimuli using physiological sensors has also been recognized in the related work [ 7 ]. This is also confirmed by the baseline results reported by the creators of the datasets, i.e., the F1-score for binary classification of low vs. high arousal and low vs. high valence are between 0.55 and 0.60 [ 26 , 56 , 57 ].…”

“…One reason is that relations between the wearable sensor data and the human psychophysiological states are not as explicit as is the relation between the wearable sensor data and human physical states. For example, smartphones can count steps and recognize human physical activities (e.g., running vs. walking) [ 6 ] but cannot recognize emotions and related affective states (e.g., cognitive load) with high accuracy [ 7 , 8 ].…”