A Deep-Learning Model for Subject-Independent Human Emotion Recognition Using Electrodermal Activity Sensors

Abstract

One of the main objectives of Active and Assisted Living (AAL) environments is to ensure that elderly and/or disabled people perform/live well in their immediate environments; this can be monitored by among others the recognition of emotions based on non-highly intrusive sensors such as Electrodermal Activity (EDA) sensors. However, designing a learning system or building a machine-learning model to recognize human emotions while training the system on a specific group of persons and testing the system on a totally a new group of persons is still a serious challenge in the field, as it is possible that the second testing group of persons may have different emotion patterns. Accordingly, the purpose of this paper is to contribute to the field of human emotion recognition by proposing a Convolutional Neural Network (CNN) architecture which ensures promising robustness-related results for both subject-dependent and subject-independent human emotion recognition. The CNN model has been trained using a grid search technique which is a model hyperparameter optimization technique to fine-tune the parameters of the proposed CNN architecture. The overall concept’s performance is validated and stress-tested by using MAHNOB and DEAP datasets. The results demonstrate a promising robustness improvement regarding various evaluation metrics. We could increase the accuracy for subject-independent classification to 78% and 82% for MAHNOB and DEAP respectively and to 81% and 85% subject-dependent classification for MAHNOB and DEAP respectively (4 classes/labels). The work shows clearly that while using solely the non-intrusive EDA sensors a robust classification of human emotion is possible even without involving additional/other physiological signals.

A Systematic Exploration of Deep Neural Networks for EDA-Based Emotion Recognition

“…ResNet and many other deep convolutional neural networks are brought up in the image domain with two-dimensional convolution operation for image feature extraction. If applying the original 2D ResNet, the 1D input signal should be rearranged to a 2D matrix, like in [11]. However, the 2D convolutional kernel will disturb the sequential distribution to extract features.…”

“…Extensive comparisons also reveal that our three deep neural networks with EDA data outperform previous models with handcraft features on emotion recognition, which proves the great potential of the end-to-end DNN method.Information 2020, 11, 212 2 of 16 different subjects. Until now, researchers still haven't achieved a satisfying recognition accuracy [8][9][10][11]. Circumventing this problem, we make great attempts on subject-independent emotion recognition in this work.Many studies in the past years have focused on physiological signals, such as EEG [12,13], ECG [14,15], and EDA [9,10].…”

“…Information 2020, 11, 212 2 of 16 different subjects. Until now, researchers still haven't achieved a satisfying recognition accuracy [8][9][10][11]. Circumventing this problem, we make great attempts on subject-independent emotion recognition in this work.…”

Towards the Applicability of Measuring the Electrodermal Activity in the Context of Process Model Comprehension: Feasibility Study

“…Notably, sweating is controlled by the sympathetic nervous system (i.e., part of the autonomic nervous system), and changes in the skin conductance are indications of physiological or psychological arousal (e.g., fight-or-flight response) [48]. Research demonstrated that this kind of arousal is significantly related to brain functions that regulate motor, sensory, and cognitive skills [44,49,50]. For example, when emotionally agitated (e.g., on the eve of an exam), sweat production is increased, resulting in an increase in the EDA as well (e.g., higher cognitive load).…”

Supervised Learning Techniques for Stress Detection in Car Drivers

“…The system described in [15] uses the time differences between consecutive R-waves of QRS complexes, also called RR intervals, for mental stress identification in firefighters. In several works, a comparison between ML and DL methods for stress recognition has been carried out, also showing that Convolutional Neural Networks (CNNs) can outperform ML conventional methods [16,17].…”