Cognitive load estimation using ocular parameters in automotive

Prabhakar, Gowdham; Mukhopadhyay, Abhishek; Murthy, L. R. D.; Madan, Modiksha; Deshmukh, Sachin; Biswas, Pradipta

doi:10.1016/j.treng.2020.100008

Cited by 40 publications

(14 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pupil area is strongly associated with the user's ongoing task difficulty [166], [132], [172] and mean pupil diameter has been shown to have a positive correlation with the cognitive workload in several different tasks [17], [170], [154]. The standard deviation of the pupil size also increases with cognitive load [173]. In a task where the participants had to watch a multimedia lesson, Zu et al trying to determine what type of cognitive load affects pupil size, concluded that mean ratio of pupil size change was most sensitive to extraneous and germane load [148].…”

Section: Metrics Related To Cognitive Workloadmentioning

confidence: 99%

Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes

Skaramagkas

Giannakakis

Ktistakis

et al. 2023

IEEE Rev. Biomed. Eng.

157

View full text Add to dashboard Cite

Eye behaviour provides valuable information revealing one's higher cognitive functions and state of affect. Although eye tracking is gaining ground in the research community, it is not yet a popular approach for the detection of emotional and cognitive states. In this paper, we present a review of eye and pupil tracking related metrics (such as gaze, fixations, saccades, blinks, pupil size variation, etc.) utilized towards the detection of emotional and cognitive processes, focusing on visual attention, emotional arousal and cognitive workload. Besides, we investigate their involvement as well as the computational recognition methods employed for the reliable emotional and cognitive assessment. The publicly available datasets employed in relevant research efforts were collected and their specifications and other pertinent details are described. The multimodal approaches which combine eye-tracking features with other modalities (e.g. biosignals), along with artificial intelligence and machine learning techniques were also surveyed in terms of their recognition/classification accuracy. The limitations, current open research problems and prospective future research directions were discussed for the usage of eyetracking as the primary sensor modality. This study aims to comprehensively present the most robust and significant eye/pupil metrics based on available literature towards the development of a robust emotional or cognitive computational model.

show abstract

Section: Metrics Related To Cognitive Workloadmentioning

confidence: 99%

Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes

Skaramagkas

Giannakakis

Ktistakis

et al. 2023

IEEE Rev. Biomed. Eng.

157

View full text Add to dashboard Cite

show abstract

“…For instance, a recent review on cognitive load estimation in driving provided eight different measures as candidates for assessing a driver's cognitive load. These measures include electroencephalography and event-related potentials, optical imaging, HR and HR variability, blood pressure, skin conductance, electromyography, thermal imaging, and pupillometry [20]. Although a recent study has attempted to estimate some of these features from video streams (e.g., estimating HR from video [83]), most of them are currently measured through physiological sensors that are not accessible while solely relying on videos.…”

Section: Existing Gapsmentioning

confidence: 99%

“…Although video streams are extremely informative, they mostly provide insights about external factors. In fact, research suggests many internal factors and states (e.g., driver's cognition) cannot be accurately detected through using only cameras [20]. For instance, a driver might smile when being frustrated, leading to a misleading inference about the driver's state [21].…”

Section: Introductionmentioning

confidence: 99%

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

et al. 2021

View full text Add to dashboard Cite

Effective shared autonomy requires a clear understanding of driver's behavior, which is governed by multiple psychophysiological and environmental variables. Disentangling this intricate web of interactions requires understanding the driver's state and behaviors in different real-world scenarios, longitudinally. Naturalistic Driving Studies (NDS) have shown to be an effective approach to understanding the driver's state and behavior in real-world scenarios. However, due to the lack of technological and computing capabilities, former NDS only focused on vision-based approaches, ignoring important psychophysiological factors such as cognition and emotion. The main objective of this paper is to introduce HARMONY, a human-centered multimodal naturalistic driving study, where driver's behaviors and states are monitored through (1) in-cabin and outside video streams (2) physiological signals including driver's heart rate and hand acceleration (IMU data), (3) ambient noise, light, and the vehicle's GPS location, and (4) music logs, including song features such as tempo. HARMONY is the first study that collects long-term naturalistic facial, physiological, and environmental data simultaneously. This paper summarizes HARMONY's goals, framework design, data collection and analysis, and the on-going and future research efforts. Through a presented case study, we first demonstrate the importance of longitudinal driver state sensing through using Kernel Density Estimation Methods. Second, we leverage the application of Bayesian Change Point detection methods to demonstrate how we can identify driver behaviors and responses to the environmental conditions by fusing psychophysiological information with features extracted from video streams.INDEX TERMS Naturalistic driving study, physiological sensing,driver state detection, shared-autonomy, contextual awareness, human-in-the-loop systems

show abstract

“…In this section, we describe a user study that was conducted to validate if L1 Norm of Spectrum (L1NS), Standard Deviation of Pupil (STDP), Low Pass Filter (LPF) of pupil diameter saccade rate, fixation rate, and median SI velocity can distinguish between different cognitive workloads of participants caused by task difficulty. Detailed description of metrics and their implementation can be found in Prabhakar et al (2020). We used psychometric tests like the N-back test and arithmetic questions to assess the increase in participants' cognitive workload with increased task difficulty.…”

Section: User Study On Psychometric Testsmentioning

confidence: 99%

Correlation Between Physiological and Performance-Based Metrics to Estimate Pilots' Cognitive Workload

Hebbar

Bhattacharya²,

Prabhakar³

et al. 2021

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

This paper discusses the utilization of pilots' physiological indications such as electroencephalographic (EEG) signals, ocular parameters, and pilot performance-based quantitative metrics to estimate cognitive workload. The study aims to derive a non-invasive technique to estimate pilot's cognitive workload and study their correlation with standard physiological parameters. Initially, we conducted a set of user trials using well-established psychometric tests for evaluating the effectiveness of pupil and gaze-based ocular metrics for estimating cognitive workload at different levels of task difficulty and lighting conditions. Later, we conducted user trials with the NALSim flight simulator using a business class Learjet aircraft model. We analyzed participants' ocular parameters, power levels of different EEG frequency bands, and flight parameters for estimating variations in cognitive workload. Results indicate that introduction of secondary task increases pilot's cognitive workload significantly. The beta frequency band of EEG, nearest neighborhood index specifying distribution of gaze fixation, L1 Norm of power spectral density of pupil diameter, and the duty cycle metric indicated variations in cognitive workload.

show abstract

Cognitive load estimation using ocular parameters in automotive

Cited by 40 publications

References 24 publications

Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes

Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

Correlation Between Physiological and Performance-Based Metrics to Estimate Pilots' Cognitive Workload

Contact Info

Product

Resources

About