A Cross-Corpus Study on Speech Emotion Recognition

Milner, Rosanna; Jalal, Asif; Ng, Raymond W. M.; Hain, Thomas

doi:10.1109/asru46091.2019.9003838

Cited by 26 publications

(22 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It was shown that there have been many studies on cross-corpus SER performance, mainly transferring the feature space or fine-tuned high-level features. Milner et al [ 63 ] extracted log-mel filterbanks with 23 dimensions and a pretrained source dataset with BiLSTM and transferred the model to learn the target dataset. Table 10 shows that our fine-tuned method outperforms the existing method in RAVDESS, EmoDB, and SAVEE.…”

Section: Resultsmentioning

confidence: 99%

Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition

Seo

2020

Sensors

View full text Add to dashboard Cite

Speech emotion recognition (SER) classifies emotions using low-level features or a spectrogram of an utterance. When SER methods are trained and tested using different datasets, they have shown performance reduction. Cross-corpus SER research identifies speech emotion using different corpora and languages. Recent cross-corpus SER research has been conducted to improve generalization. To improve the cross-corpus SER performance, we pretrained the log-mel spectrograms of the source dataset using our designed visual attention convolutional neural network (VACNN), which has a 2D CNN base model with channel- and spatial-wise visual attention modules. To train the target dataset, we extracted the feature vector using a bag of visual words (BOVW) to assist the fine-tuned model. Because visual words represent local features in the image, the BOVW helps VACNN to learn global and local features in the log-mel spectrogram by constructing a frequency histogram of visual words. The proposed method shows an overall accuracy of 83.33%, 86.92%, and 75.00% in the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS), the Berlin Database of Emotional Speech (EmoDB), and Surrey Audio-Visual Expressed Emotion (SAVEE), respectively. Experimental results on RAVDESS, EmoDB, SAVEE demonstrate improvements of 7.73%, 15.12%, and 2.34% compared to existing state-of-the-art cross-corpus SER approaches.

show abstract

Section: Resultsmentioning

confidence: 99%

Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition

Seo

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…[3] introduce a domain-adaptive subspace learning method to reduce the feature space differences between source and target speech. In [4] a bidirectional LSTM with attention mechanism is used for classifying emotions across various corpora. To generalize to emotions across corpora authors suggest to use models trained on out-of-domain data and conduct adaptation to the missing corpus or use domain adversarial training (DAT) [5].…”

Section: Related Workmentioning

confidence: 99%

A study on cross-corpus speech emotion recognition and data augmentation

Braunschweiler¹,

Doddipatla²,

Keizer³

et al. 2022

Preprint

View full text Add to dashboard Cite

Models that can handle a wide range of speakers and acoustic conditions are essential in speech emotion recognition (SER). Often, these models tend to show mixed results when presented with speakers or acoustic conditions that were not visible during training. This paper investigates the impact of cross-corpus data complementation and data augmentation on the performance of SER models in matched (test-set from same corpus) and mismatched (test-set from different corpus) conditions. Investigations using six emotional speech corpora that include single and multiple speakers as well as variations in emotion style (acted, elicited, natural) and recording conditions are presented. Observations show that, as expected, models trained on single corpora perform best in matched conditions while performance decreases between 10-40% in mismatched conditions, depending on corpus specific features. Models trained on mixed corpora can be more stable in mismatched contexts, and the performance reductions range from 1 to 8% when compared with single corpus models in matched conditions. Data augmentation yields additional gains up to 4% and seem to benefit mismatched conditions more than matched ones.

show abstract

“…One of the possible reasons why attention models outperform others is that the models learn the biases for a specific task, or group of tasks, leading to improved generalisation. Recently, a sequence and attention-based domain adversarial system was presented in [20] which investigated whether the information in acted datasets can be learnt to benefit emotion prediction for natural datasets and achieved state-of-the-art results.…”

Section: Related Workmentioning

confidence: 99%

“…The utterances are split into a training set of 4290 samples (Sessions 1-4) and a test set of 1241 samples(Session 5). This is referred to as IEM4 in this paper and in [20].…”

Section: Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Removing Bias with Residual Mixture of Multi-View Attention for Speech Emotion Recognition

et al. 2020

Self Cite

View full text Add to dashboard Cite

Speech emotion recognition is essential for obtaining emotional intelligence which affects the understanding of context and meaning of speech. The fundamental challenges of speech emotion recognition from a machine learning standpoint is to extract patterns which carry maximum correlation with the emotion information encoded in this signal, and to be as insensitive as possible to other types of information carried by speech. In this paper, a novel recurrent residual temporal context modelling framework is proposed. The framework includes mixture of multi-view attention smoothing and high dimensional feature projection for context expansion and learning feature representations. The framework is designed to be robust to changes in speaker and other distortions, and it provides state-of-the-art results for speech emotion recognition. Performance of the proposed approach is compared with a wide range of current architectures in a standard 4-class classification task on the widely used IEMOCAP corpus. A significant improvement of 4% unweighted accuracy over state-of-the-art systems is observed. Additionally, the attention vectors have been aligned with the input segments and plotted at two different attention levels to demonstrate the effectiveness.

show abstract

A Cross-Corpus Study on Speech Emotion Recognition

Cited by 26 publications

References 29 publications

Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition

Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition

A study on cross-corpus speech emotion recognition and data augmentation

Removing Bias with Residual Mixture of Multi-View Attention for Speech Emotion Recognition

Contact Info

Product

Resources

About