Self-labeling with feature transfer for speech emotion recognition

Wen, Gongjian; Liao, Huiqiang; Li, Huihui; Wen, Pengcheng; Zhang, Tong; Gao, Sande; Wang, Bao

doi:10.1016/j.knosys.2022.109589

Cited by 11 publications

(7 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Manohar and Logashanmugam [9] integrated different meta-heuristic and deep-learning methods for SER with selected features. Wen et al [10] Introduced self-labeling feature frames in their DLbased SER study.…”

Section: Literature Reviewmentioning

confidence: 99%

Emotion Recognition with Intensity Level from Bangla Speech using Feature Transformation and Cascaded Deep Learning Model

Billah,

Sarker,

Akhand

et al. 2024

IJACSA

View full text Add to dashboard Cite

Speech Emotion Recognition (SER) identifies and categorizes emotional states by analyzing speech signals. The intensity of specific emotional expressions (e.g., anger) conveys critical directives and plays a crucial role in social behavior. SER is intrinsically language-specific; this study investigated a novel cascaded deep learning (DL) model to Bangla SER with intensity level. The proposed method employs the Mel-Frequency Cepstral Coefficient, Short-Time Fourier Transform (STFT), and Chroma STFT signal transformation techniques; the respective transformed features are blended into a 3D form and used as the input of the DL model. The cascaded model performs the task in two stages: classify emotion in Stage 1 and then measure the intensity in Stage 2. DL architecture used in both stages is the same, which consists of a 3D Convolutional Neural Network (CNN), a Time Distribution Flatten (TDF) layer, a Long Short-term Memory (LSTM), and a Bidirectional LSTM (Bi-LSTM). CNN first extracts features from 3D formed input; the features are passed through the TDF layer, Bi-LSTM, and LSTM; finally, the model classifies emotion along with its intensity level. The proposed model has been evaluated rigorously using developed KBES and other datasets. The proposed model revealed as the best-suited SER method compared to existing prominent methods achieving accuracy of 88.30% and 71.67% for RAVDESS and KBES datasets, respectively.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Emotion Recognition with Intensity Level from Bangla Speech using Feature Transformation and Cascaded Deep Learning Model

Billah,

Sarker,

Akhand

et al. 2024

IJACSA

View full text Add to dashboard Cite

show abstract

“…Te loss function widely used in speech emotion recognition is the cross-entropy loss (CEL) [2]. Te center loss function [41] is also used for SER to pull features in the same emotional category to its center [40].…”

Section: Loss Functionmentioning

confidence: 99%

“…Speech not only explicitly expresses linguistic content but also implicitly contains the speaker's emotional states such as sadness, happiness, and fear. Te speech emotion recognition (SER) aims to automatically identify the speaker's emotional states [1][2][3], having a large number of applications such as human-computer interaction, information recommendation, and health detection. Consequently, methods for SER are deeply investigated.…”

Section: Introductionmentioning

confidence: 99%

“…Te emotional labels of speech may be uncertain [1] for those methods based on frames for SER. When they segment each speech sentence into frames, the label of each frame is assigned by that of the sentence, easily leading to the noise labels [2]. Some new methods are proposed to solve this problem, such as the iterative self-learning framework that designed four specifc label change rules [31] and the selflabeling method for each speech frame [2].…”

Section: Introductionmentioning

confidence: 99%

“…When they segment each speech sentence into frames, the label of each frame is assigned by that of the sentence, easily leading to the noise labels [2]. Some new methods are proposed to solve this problem, such as the iterative self-learning framework that designed four specifc label change rules [31] and the selflabeling method for each speech frame [2]. Simultaneously, the multiclassifer mutual learning is also proposed [1], where all classifers classify each sample and then combine their classifcation results to construct its new label.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multimodal and Multitask Learning with Additive Angular Penalty Focus Loss for Speech Emotion Recognition

Wen,

Ye,

et al. 2023

International Journal of Intelligent Systems

Self Cite

View full text Add to dashboard Cite

Speech emotion recognition has lots of applications such as human-computer interaction and health management. The current methods are challenged with the problems of fuzzy decision boundary and imbalance between difficult and easy samples in the training data. This paper first proposes an additive angle penalty focus loss function (APFL), which strictly refines the fuzzy decision boundary by introducing angle penalty factors to improve the compactness within the class and enlarge the distance between classes. It also assigns the larger loss to difficult samples to make the model pay more attention to them, as they are easily misclassified. Simultaneously, due to the lack of training samples, the framework of multimodal and multitask learning with APFL is further proposed, which extracts spectrogram features by deep neural network, text features by the pretrained language model, and audio features by the pretrained sound model. It uses the gender recognition as an auxiliary task. The experimental results verify the effectiveness of the proposed loss function and framework.

show abstract

Diversity subspace generation based on feature selection for speech emotion recognition

Ye,

Sun

2023

Multimed Tools Appl

View full text Add to dashboard Cite

Self-labeling with feature transfer for speech emotion recognition

Cited by 11 publications

References 67 publications

Emotion Recognition with Intensity Level from Bangla Speech using Feature Transformation and Cascaded Deep Learning Model

Emotion Recognition with Intensity Level from Bangla Speech using Feature Transformation and Cascaded Deep Learning Model

Multimodal and Multitask Learning with Additive Angular Penalty Focus Loss for Speech Emotion Recognition

Diversity subspace generation based on feature selection for speech emotion recognition

Contact Info

Product

Resources

About