EEG-Based Affect and Workload Recognition in a Virtual Driving Environment for ASD Intervention

Abstract

objective To build group-level classification models capable of recognizing affective states and mental workload of individuals with autism spectrum disorder (ASD) during driving skill training. Methods Twenty adolescents with ASD participated in a six-session virtual reality driving simulator based experiment, during which their electroencephalogram (EEG) data were recorded alongside driving events and a therapist’s rating of their affective states and mental workload. Five feature generation approaches including statistical features, fractal dimension features, higher order crossings (HOC)-based features, power features from frequency bands, and power features from bins (Δf = 2 Hz) were applied to extract relevant features. Individual differences were removed with a two-step feature calibration method. Finally, binary classification results based on the k-nearest neighbors algorithm and univariate feature selection method were evaluated by leave-one-subject-out nested cross-validation to compare feature types and identify discriminative features. Results The best classification results were achieved using power features from bins for engagement (0.95) and boredom (0.78), and HOC-based features for enjoyment (0.90), frustration (0.88), and workload (0.86). Conclusion Offline EEG-based group-level classification models are feasible for recognizing binary low and high intensity of affect and workload of individuals with ASD in the context of driving. However, while promising the applicability of the models in an online adaptive driving task requires further development. Significance The developed models provide a basis for an EEG-based passive brain computer interface system that has the potential to benefit individuals with ASD with an affect- and workload-based individualized driving skill training intervention.

Physiological-signal-based mental workload estimation via transfer dynamical autoencoders in a deep learning framework

“…It is shown the average performance of the proposed method is competitive against our previous one. Such MW recognition accuracy is also comparable with the newest work reported in [32] , where 0.86 workload classification rate is achieved (note that the environment for implementing classifier is different in two works). However, the subjectspecific classification paradigm is used in [11] while we trained subject-generic classifiers for TDAE.…”

“…Guler and Ubeyli fused the discrete wavelet transformation and the adaptive neuron fuzzy inference system as a multiclass EEG classifier [31] . In a newly reported work [32] , the classical k -nearest neighbor method also showed high performance. In our previous works [16,33] , we applied the least square support vector machine (LSSVM) and the locally linear embedding to classify operator MW.…”

Virtual reality exposure therapy for fear of driving: analysis of clinical characteristics, physiological response, and sense of presence

“…Despite these interesting results, our study has several limitations, such as the small sample size; use of nonparametric statistical methods; and the fact that we did not record other physiological parameters of interest, such as respiratory rate and electroencephalography. 50 In addition, it is known that more realistic forms of VR can elicit more intense experiences of awe. Towards this end, we intend to improve the software and apparatus used in future studies.…”

Assessing cognitive mental workload via EEG signals and an ensemble deep learning classifier based on denoising autoencoders

“…The EEG signals were rerecorded to study the MWs of the crew members during cooperative tasks [19]. Based on EEG data and therapist's evaluation during driving skill training, Fan et al established a group-level classification model to distinguish the emotional states and MW of patients in the autism spectrum disorder during driving skill training [20]. Chen et al demonstrated the validity of EEG signals in evaluating the MW of construction workers and described the development of a wearable EEG safety helmet prototype [21].…”