Emotion Recognition on Facial Expression and Voice: Analysis and Discussion

Abstract

Emotion plays an important role in our daily lives. Emotional individuals can affect the performance of a company, the harmony of a family, the wellness or growth (physical, mental, and spiritual) of a child etc. It renders a wide range of impacts. The existing works on emotion detection from facial expressions differ from the voice. It is deduced that the facial expression is captured on the face externally, whereas the voice is captured from the air passes through the vocal folds internally. Both captured output models may very much deviate from each other. This paper studies and analyses a person's emotion through dual models -- facial expression and voice separately. The proposed algorithm uses a Convolutional Neural Network (CNN) with 2-dimensions convolutional layers for facial expression and 1-Dimension convolutional layers for voice. Feature extraction is done via face detection, and Mel-Spectrogram extraction is done via voice. The network layers are fine-tuned to achieve the higher performance of the CNN model. The trained CNN models can recognize emotions from the input videos, which may cover single or multiple emotions from the facial expression and voice perspective. The experimented videos are clean from the background music and environment noise and contain only a person's voice. The proposed algorithm achieved an accuracy of 62.9% through facial expression and 82.3% through voice.

How perceived lack of benevolence harms trust of artificial intelligence management.

“…Gray & Wegner, 2012;Huebner, 2010;Kramer et al, 2018;Searle, 1980). AI can recognize and mimic human emotions using techniques such as convolutional neural networks for facial expression analysis (Kajale et al, 2023;Ng et al, 2023). However, AI's identification and expression of emotions is based on pattern recognition and data analysis rather than genuine emotional comprehension through "the experience of a sentient being" (Sajjad et al, 2023, p. 820).…”

“…We did not directly measure the capability of AI management to experience emotions in our studies. Although prior work has shown that AI is unable to experience emotions and that individuals recognize this limitation (Bigman & Gray, 2018;Kajale et al, 2023;Ng et al, 2023;Sajjad et al, 2023), researchers have begun to explore factors, such as anthropomorphism, that might increase the perceived emotional experience of AI. Although some studies suggest that making AI managers more humanlike might increase trust, others suggest that attempting to make them too humanlike can lead to backlash (Mori et al, 2012;Schroeder & Epley, 2016;Schroeder & Schroeder, 2018).…”

“…Future research may also explore individual differences which might attenuate the perception that AI is deficient in its ability to experience emotions (Bigman & Gray, 2018;Kajale et al, 2023;Ng et al, 2023). Characteristics of employees (e.g., gender, age, race, culture) might influence the inferred emotional experience of AI management, its perceived benevolence, and the subsequent effect on trust.…”