Emotional Human-Machine Interaction: Cues from Facial Expressions

Abstract

Emotion detection provides a promising basis for designing futureoriented human centered design of Human-Machine Interfaces. Affective Computing can facilitate human-machine communication. Such adaptive advanced driver assistance systems (ADAS) which are dependent on the emotional state of the driver can be applied in cars. In contrast to the majority of former studies that only used static recognition methods, we investigated a new dynamic approach for detecting emotions in facial expressions in an artificial setting and in a driving context. By analyzing the changes of an area defined by a number of dots that were arranged on participants' faces, variables were extracted to classify the participants' emotions according to the Facial Action Coding System. The results of our novel way to categorize emotions lead to a discussion on additional applications and limitations that frames an attempted approach of emotion detection in cars. Implications for further research and applications are outlined.

Deep Neural Network Approach for Pose, Illumination, and Occlusion Invariant Driver Emotion Detection

“…In 2012 [ 23 ], Alvarez et al introduced emotional adaptive vehicle user interfaces using logistic model trees, multilayer perceptron, naive Bayes, and logistic regression methods to predict the driver’s emotions using speech signals. In 2011, Tews et al [ 24 ] proposed an emotional human–machine interaction system using a statistical variance method to predict the emotions from the face. Paschero et al [ 25 ] in 2012 introduced a real-time classifier for a vehicle driver’s emotion recognition from the face using the multi-layer perceptron method to classify emotions like happiness, anger, fear, sadness, disgust, and surprise.…”

“…Among all these works, some results [ 15 , 25 , 26 , 28 , 31 , 33 ] have proposed systems running in a non-car environment, whereas works [ 20 , 29 , 37 , 40 , 41 , 42 ] have been conducted in a real-time environment. Some results [ 14 , 16 , 17 , 18 , 24 , 30 , 38 , 39 ] have used a simulator environment.…”

Developments in Implicit Measurements

“…Therefore, detecting and mitigating driver emotions by using affective computing in an emotion-aware system may ensure driving safety (Ihme et al, 2019). One idea of such a system is to interpret the user's emotional state and provide assistance to support users to reduce the negative consequences of certain emotional states (Klein et al, 2002;Tews et al, 2011;Jeon, 2015;Löcken et al, 2017;Ihme et al, 2018). Furthermore, in the context of high-level automated driving functions, an automated assessment of emotions could allow adapting driving styles or warnings to the drivers' current emotional state to maximize drivers' comfort and optimize the driving experience (Techer et al, 2019).…”

Understanding the Multidimensional and Dynamic Nature of Facial Expressions Based on Indicators for Appraisal Components as Basis for Measuring Drivers' Fear

“…Therefore, detecting and mitigating driver emotions by using affective computing in an emotion-aware system may ensure driving safety (Ihme et al, 2019 ). One idea of such a system is to interpret the user's emotional state and provide assistance to support users to reduce the negative consequences of certain emotional states (Klein et al, 2002 ; Tews et al, 2011 ; Jeon, 2015 ; Löcken et al, 2017 ; Ihme et al, 2018 ). Furthermore, in the context of high-level automated driving functions, an automated assessment of emotions could allow adapting driving styles or warnings to the drivers' current emotional state to maximize drivers' comfort and optimize the driving experience (Techer et al, 2019 ).…”