Automatic Speech Emotion Recognition of Younger School Age Children

Matveev, Yuri; Matveev, Anton; Frolova, Оlga; Lyakso, Elena E.; Nersisson, Ruban

doi:10.3390/math10142373

Cited by 13 publications

(4 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the difficulties in obtaining emotions data, there are corpora of children's emotional speech in different languages [24][25][26][27] and emotional facial expressions [28] of children. Research is being conducted on the automatic recognition of emotions from children's speech [29,30] and their facial expressions [31]. The accuracy of emotion recognition can be higher when using several modalities [32], for example audio and video, which requires the collection of appropriate audio-visual corpora.…”

Section: Children's Audio-visual Speech Emotion Corporamentioning

confidence: 99%

A Neural Network Architecture for Children’s Audio-Visual Emotion Recognition

Matveev,

Frolova

et al. 2023

Preprint

View full text Add to dashboard Cite

Detecting and understanding emotions is critical for our daily activities. As emotion recognition (ER) systems develop, we start looking at more difficult cases than just acted adult audio-visual speech. In this work, we investigate automatic classification of audio-visual emotional speech of children. Our interest is, specifically, in the improvement of the utilization of the cross-modal relationships between the selected modalities: video and audio. To underscore the importance of developing ER systems for the real-world environment, we present a corpus of children’s emotional audio-visual speech that we collected. We select a state-of-the-art model as a baseline for the purposes of comparison and present several modifications focused on a deeper learning of the cross-modal relationships. By conducting experiments with our proposed approach and the selected baseline model, we observe a relative improvement in performance by 2%. Finally, we conclude that focusing more on the cross-modal relationships may be beneficial for building ER systems for child-machine communications and the environments where qualified professionals work with children.

show abstract

Section: Children's Audio-visual Speech Emotion Corporamentioning

confidence: 99%

A Neural Network Architecture for Children’s Audio-Visual Emotion Recognition

Matveev,

Frolova

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Section: Children's Audio-visual Speech Emotion Corporamentioning

confidence: 99%

A Neural Network Architecture for Children’s Audio–Visual Emotion Recognition

Matveev,

Frolova

et al. 2023

Mathematics

Self Cite

View full text Add to dashboard Cite

Detecting and understanding emotions are critical for our daily activities. As emotion recognition (ER) systems develop, we start looking at more difficult cases than just acted adult audio–visual speech. In this work, we investigate the automatic classification of the audio–visual emotional speech of children, which presents several challenges including the lack of publicly available annotated datasets and the low performance of the state-of-the art audio–visual ER systems. In this paper, we present a new corpus of children’s audio–visual emotional speech that we collected. Then, we propose a neural network solution that improves the utilization of the temporal relationships between audio and video modalities in the cross-modal fusion for children’s audio–visual emotion recognition. We select a state-of-the-art neural network architecture as a baseline and present several modifications focused on a deeper learning of the cross-modal temporal relationships using attention. By conducting experiments with our proposed approach and the selected baseline model, we observe a relative improvement in performance by 2%. Finally, we conclude that focusing more on the cross-modal temporal relationships may be beneficial for building ER systems for child–machine communications and environments where qualified professionals work with children.

show abstract

“…In the last decade, statistical machine learning models, such as Support Vector Machines (SVMs), were commonly used in this research field and they are still a vital part of many research studies [ 11 , 12 , 13 ]. At the beginning of the 21st century, Hidden Markov Models [ 14 , 15 ] were also used to explore this field of research.…”

Section: Introductionmentioning

confidence: 99%

Speech Emotion Recognition Using Attention Model

Singh

Saheer

Faust

2023

IJERPH

View full text Add to dashboard Cite

Speech emotion recognition is an important research topic that can help to maintain and improve public health and contribute towards the ongoing progress of healthcare technology. There have been several advancements in the field of speech emotion recognition systems including the use of deep learning models and new acoustic and temporal features. This paper proposes a self-attention-based deep learning model that was created by combining a two-dimensional Convolutional Neural Network (CNN) and a long short-term memory (LSTM) network. This research builds on the existing literature to identify the best-performing features for this task with extensive experiments on different combinations of spectral and rhythmic information. Mel Frequency Cepstral Coefficients (MFCCs) emerged as the best performing features for this task. The experiments were performed on a customised dataset that was developed as a combination of RAVDESS, SAVEE, and TESS datasets. Eight states of emotions (happy, sad, angry, surprise, disgust, calm, fearful, and neutral) were detected. The proposed attention-based deep learning model achieved an average test accuracy rate of 90%, which is a substantial improvement over established models. Hence, this emotion detection model has the potential to improve automated mental health monitoring.

show abstract

Automatic Speech Emotion Recognition of Younger School Age Children

Cited by 13 publications

References 64 publications

A Neural Network Architecture for Children’s Audio-Visual Emotion Recognition

A Neural Network Architecture for Children’s Audio-Visual Emotion Recognition

A Neural Network Architecture for Children’s Audio–Visual Emotion Recognition

Speech Emotion Recognition Using Attention Model

Contact Info

Product

Resources

About