Towards voice based prediction and analysis of emotions in ASD children

Abstract

Voice processing has proven to be an eminent way of recognizing the various emotions of the people. The objective of this research is to identify the presence of Autism Spectrum Disorder (ASD) and to analyze the emotions of autistic children through their voices. The presented automated voice-based system can detect and classify seven basic emotions (anger, disgust, neutral, happiness, calmness, fear and sadness) expressed by children through source parameters associated with their voices. Various prime voice features such as Mel-frequency Cepstral Coefficients (MFCC) and Spectrogram are extracted and utilized to train a Multi-layer Perceptron (MLP) Classifier to identify possible emotions exhibited by the children thereby assessing their behavioral state. This proposed work therefore helps in the examination of emotions in autistic children that can be used to assess the kind of training and care required to enhance their lifestyle.

Automatic Emotion Recognition in Children with Autism: A Systematic Literature Review

“…Apart from the emotions presented above, calmness (1), hunger (1), and nerves (1) are also recognized in those papers. Hunger and nerves are both recognized in the same study [37], and calmness is recognized in [36].…”

“…The emotions were grouped regarding their similarity. [21,24] attention (1): [25] bored (1): [25] fear-related 16 fear ( 12): [17,[26][27][28][29][30][31][32][33][34][35][36] anxiety (3): [20,22,25] trepidation (1): [37] joy-related 21 joy ( 5): [21,[26][27][28][29] liking (3): [19,20,22] happiness (12): [17,24,25,[30][31][32][33][34][35][37][38][39] smile (1): [40] relaxation (1): [37] There are studies that treat attention, engagement, involvement, and boredom as emotional states. They are more like mental states; however, they are emotion-related and were used in the context of emotion recognition in the analyzed papers.…”

“…joy 21 [17,[19][20][21][22][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39] anger 12 [17,[27][28][29][30][31][32][33]35,36,38,39] fear 12 [17,[26][27][28][29][30][31][32][33][34][35][36] disgust 7 [17,25,27,29,32,35,36] sadness 16 [17,…”

“…Number of Papers Papers joy 3 [19,20,22] joy, neutral 2 [21,24] joy, sadness 2 [21,37] joy, fear, sadness 2 [26,34] joy, disgust, sadness, neutral 1 [25] joy, anger, sadness, neutral 2 [38,39] joy, anger, fear, sadness, neutral 2 [28,30] joy, anger, fear, disgust, sadness, surprise 1 [17] joy, anger, fear, sadness, surprise, neutral 2 [31,33] joy, anger, fear, disgust, sadness, neutral 1 [36] joy, anger, fear, disgust, sadness, surprise, netural 4 [27,29,32,35] In some papers, only two-class recognition was performed. In those papers, joy is recognized as a class combined with no-joy in three papers, with a neutral state in two papers, and with sadness in two papers.…”

“…Therefore, other vocalisations and physiological responses of the nervous system are more frequently used. depth video [29,31] images [39] EMG [40] body posture RGB video [23,42] depth video [42] eye gaze RGB video [17] head movement RGB video [23,28] gestures RGB video [23,28,42] depth video [42] motion RGB video [17,23,42] depth video [17,42] sound prosody of speech audio [17,23,25,30,36,41] vocalization audio [17,23,25,36,41] heart activity heart rate ECG [19][20][21][22]43] BVP [21,23,24] variability HRV ECG [19,20,22,43] BVP [21,24] muscle activities muscle tension EMG [19][20][21]…”