Assessing Cognitive Workload Using Cardiovascular Measures and Voice

Abstract

Monitoring cognitive workload has the potential to improve both the performance and fidelity of human decision making. However, previous efforts towards discriminating further than binary levels (e.g., low/high or neutral/high) in cognitive workload classification have not been successful. This lack of sensitivity in cognitive workload measurements might be due to individual differences as well as inadequate methodology used to analyse the measured signal. In this paper, a method that combines the speech signal with cardiovascular measurements for screen and heartbeat classification is introduced. For validation, speech and cardiovascular signals from 97 university participants and 20 airline pilot participants were collected while cognitive stimuli of varying difficulty level were induced with the Stroop colour/word test. For the trinary classification scheme (low, medium, high cognitive workload) the prominent result using classifiers trained on each participant achieved 15.17 ± 0.79% and 17.38 ± 1.85% average misclassification rates indicating good discrimination at three levels of cognitive workload. Combining cardiovascular and speech measures synchronized to each heartbeat and consolidated with short-term dynamic measures might therefore provide enhanced sensitivity in cognitive workload monitoring. The results show that the influence of individual differences is a limiting factor for a generic classification and highlights the need for research to focus on methods that incorporate individual differences to achieve even better results. This method can potentially be used to measure and monitor workload in real time in operational environments.

Collaborative System Usability in Spaceflight Analog Environments through Remote Observations

“…Work by Li and colleagues [12] suggests that trust in autonomous agents can be inferred from lexical and acoustic features in speech. Similarly, Magnusdottir et al [13] coupled cardiovascular measures and speech to measure the dynamics of cognitive workload during various tasks (see also [14]). Voice recordings may serve as a non-intrusive measure of critical dynamics in human-machine interactions.…”

Cognitive Load Assessment of Air Traffic Controller Based on SCNN-TransE Network Using Speech Data

“…Lastly, physiological evaluation methods conduct an evaluation of a controller's cognitive load by measuring the changes in physiological indicators produced by the controller during their work. The most commonly used indices are the cardiac activity index, eye movement analysis index, electroencephalogram (EEG) analysis index, and speech analysis index [17][18][19][20][21][22][23][24][25]. These methods use a variety of tools and equipment to measure the physiological indicators of ATCs to ensure the objective and real-time nature of the data collected.…”