Speech-based cognitive load monitoring system

Yin, Bo; Chen, Fang; Ruiz, Natalie; Ambikairajah, Eliathamby

doi:10.1109/icassp.2008.4518041

Cited by 75 publications

(94 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The major challenge of choosing the assessment features for automated cognitive load detection is to make sure they satisfy the requirements of consistency, compact representation and automatic acquisition [98]. Cognitive load measurement approaches aim to find effective features which reliably reflect the cues and can be extracted automatically such that they are useful in adaptive systems.…”

Section: Estimating Load From Interactive Behaviormentioning

confidence: 99%

“…Cognitive load measurement approaches aim to find effective features which reliably reflect the cues and can be extracted automatically such that they are useful in adaptive systems. They also aim to find a suitable learning or modeling scheme for each index in order to resolve the corresponding level of cognitive load [98]. By manipulating the level of task complexity and cognitive load, and conducting a series of repeated measures user studies in a variety of scenarios, it has been able to identify a series of cognitive load indices based on features from a number of input modalities, specifically, observations of significant changes in speech and digital-pen input that are abstracted from individual application domains in which they occur as well as correlated to high cognitive load.…”

Section: Estimating Load From Interactive Behaviormentioning

confidence: 99%

See 1 more Smart Citation

The State-of-The-Art

Chen¹,

Zhou²,

Wang³

et al. 2016

Robust Multimodal Cognitive Load Measurement

Self Cite

View full text Add to dashboard Cite

Recent years have witnessed booming growth in the research of cognitive load understanding and measurement [1]. Extensive research has been carried out into cognitive load theory, cognitive load measurement, applications of cognitive load theory, and the factors that may affect cognitive load. This chapter surveys the state-of-the-art of cognitive load measurement and gives an overview of current research in cognitive load measurement. The highlights include:

show abstract

Section: Estimating Load From Interactive Behaviormentioning

confidence: 99%

Section: Estimating Load From Interactive Behaviormentioning

confidence: 99%

The State-of-The-Art

Chen¹,

Zhou²,

Wang³

et al. 2016

Robust Multimodal Cognitive Load Measurement

Self Cite

View full text Add to dashboard Cite

show abstract

“…pitch, prosody, pauses, and disfluencies, have also been found to be changing under high levels of CL [4,[8][9][10]. Such measures allow non-intrusive analysis as they are based on speech data generated by users while they complete the task.…”

Section: Introductionmentioning

confidence: 99%

Cognitive Load Measurement from User’s Linguistic Speech Features for Adaptive Interaction Design

Khawaja

Chen

Owen

et al. 2009

Human-Computer Interaction – INTERACT 2009

Self Cite

View full text Add to dashboard Cite

Abstract. An adaptive interaction system, which is aware of the user's current cognitive load (CL), can change its response, presentation and flow of interaction material accordingly, to improve user's experience and performance. We present a speech content analysis approach to CL measurement, which employs users' linguistic features of speech to determine their experienced CL level. We show analyses of several linguistic features, extracted from speech of personnel working in computerized incident control rooms and involved in highly complex bushfire management tasks in Australia. We present the results of linguistic features showing significant differences between the speech from the cognitively low load and high load tasks. We also discuss how the method may be used for user interface evaluation and interaction design improvement.

show abstract

“…Speech processing offers the ability to monitor cognitive workload in a non-intrusive way, compared to measures such as blood pressure, heart rate, electroencephalogram or electrocardiogram. Measuring voice is easy and recent work has shown very promising results in classifying cognitive workload levels using the speech signal [4][5][6]. Successful design and implementation of such a method would provide a powerful tool to developers of cognitive infocommunications systems [7].…”

Section: Introductionmentioning

confidence: 99%

“…There is a growing body of work that supports the statement that cognitive workload affects the speech signal. For example, a data set of 15 participants was presented in [4] where each participant performed reading and Stroop tasks. The difficulty levels of these tasks were classified using mel-frequency cepstrum coefficients and prosodic features and a classifier based on a speaker-adapted Gaussian mixture model.…”

Section: Introductionmentioning

confidence: 99%

Monitoring Cognitive Workload Using Vocal Tract and Voice Source Features

Magnúsdóttir

Borský

Meier

et al. 2017

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

IntroductionMonitoring cognitive workload in individuals that are performing safety-critical jobs has huge potential, for example in aviation [1]. The relationship between cognitive workload and performance has been well studied [2] and the connection between cognitive workload and physical health has also been highlighted [3]. Clearly, it is crucial to manage cognitive workload in the modern day work environment, both in terms of performance and health. Speech processing offers the ability to monitor cognitive workload in a non-intrusive way, compared to measures such as blood pressure, heart rate, electroencephalogram or electrocardiogram. Measuring voice is easy and recent work has shown very promising results in classifying cognitive workload levels using the speech signal [4][5][6]. Successful design and implementation of such a method would provide a powerful tool to developers of cognitive infocommunications systems [7].The main objective of this work is to provide an independent verification of whether there is a link between increased cognitive workload and changes in the speech signal and, if this link exists, to characterize what part of voice is mostly affected. To do this we conducted a cognitive workload experiment where the set tasks were primarily solved using speech from 98 participants. Each participant read a standard passage of text on a computer screen and solved Stroop tasks with three difficulty levels [8]. A separate classifier of the three difficulty levels was trained for each speaker based on two sets of voice parameters comprising of the vocal tract (VT) and voice source (VS) features respectively. The conclusion is that task load does affect the speech signal and that vocal tract parameters are a better indicator of task level than voice source parameters. The context of the work is presented in Section 2 and the speech processing approach is described in Section 3. The experimental methodology and detailed results are given in Section 4 and 5. The results are summarized in Section 6 and the discussion is continued in Section 7.

show abstract

Speech-based cognitive load monitoring system

Cited by 75 publications

References 5 publications

The State-of-The-Art

The State-of-The-Art

Cognitive Load Measurement from User’s Linguistic Speech Features for Adaptive Interaction Design

Monitoring Cognitive Workload Using Vocal Tract and Voice Source Features

Contact Info

Product

Resources

About