Cognitive Load Monitoring With Wearables–Lessons Learned From a Machine Learning Challenge

Abstract

To further extend the applicability of wearable sensors, methods for accurately extracting subtle psychological information from the sensor data are required. However, accessing subjective information in everyday life, such as cognitive load, remains challenging. To bring consensus on methods for cognitive load monitoring, a machine learning challenge is organized. The participants developed machine learning methods for cognitive load classification using wrist-worn physiological sensors' data, namely heart rate, R-R intervals, skin conductance, and skin temperature. The data from subjects solving cognitive tasks of varying difficulty was used for the challenge. This article presents a systematic comparison and multistrategic performance evaluation of the thirteen methods submitted to this challenge. A systematic comparison of preprocessing, classifiers, and implementation techniques is presented. Performance variations for different task difficulty levels, different subjects, and different experiment periods are evaluated. The results indicate that the most robust methods used multimodal sensor data, classical classification approaches such as decision trees and support vector machines or their ensembles, and Bayesian hyperparameter optimization for hyperparameter tuning. The most accurate models used handcrafted features that are further selected using sequential backward floating search and evaluated using stratified person-aware crossvalidation strategy. Moreover, the results indicated better classification performance for specific test subjects, the tasks with the highest difficulty, and in some cases, the time elapsed since the start of the experiment. This dependency is likely due to model overfitting or due to the subjective nature of the psychophysiological process. The intersubject variability in responses is challenging to be captured through objective binary labels for cognitive load, thereby warranting more sophisticated annotation approaches.

Machine Learning models for the Cognitive Stress Detection Using Heart Rate Variability Signals

“…While the classification accuracy in this study is slightly lower when compared to prior studies [3,25,54], our models can be considered acceptable. Using the same dataset to infer cognitive load from a wristworn physiological sensor's data, Gjoreski et al, [54] reported the accuracies of 13 machine-learning methods ranging from 0.50 to 0.69. Using a wearable device, Tervonen et al, [3] compared six ultra-short window length measurements (5, 10, 25, and 30 s) to detect cognitive load.…”

Predicting Emotion with Biosignals: A Comparison of Classification and Regression Models for Estimating Valence and Arousal Level Using Wearable Sensors

“…Moreover, feature selection needs to be studied in order to provide reliable estimates for each individual. For instance, a sequential backward floating search has been found to be an effective feature selection method for biosignals [ 34 ].…”

“…Person-specific z -score normalization has been found to be the most effective way to normalize biosignals [ 33 , 34 ] when classifying affect and stress stages. Due to this, it was used in this study.…”

IoT-Enabled Remote Monitoring Techniques for Healthcare Applications -- An Overview

“…The techniques involved in the remote monitoring of patients or individuals have expanded in various implementations -for example, telemedicine, clinical trials, online counseling, psychological monitoring [43], and so forth, have diverged the markets using wearables, mobile phones, or implantable sensor units. Particularly, the focus on the remote monitoring of the health status of patients/individuals involved elderly [61,75], chronic patients, or preventive care individuals such as working professionals.…”