Analyzing the memory of BLSTM Neural Networks for enhanced emotion classification in dyadic spoken interactions

Wöllmer, Martin; Metallinou, Angeliki; Katsamanis, Nassos; Schuller, Björn; Narayanan, Shrikanth

doi:10.1109/icassp.2012.6288834

Cited by 21 publications

(23 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, newly reconstructed data are obtained by applying this rule on the emotion-specific data in a different domain. Wollmer et al [41] presents a method to systematically investigate the number of past and future utterance-level observations that are considered to generate an emotion prediction for a given utterance, and to examine to what extent this temporal bidirectional context contributes to the overall performance.…”

Section: B Feature Extractionmentioning

confidence: 99%

Learning Salient Features for Speech Emotion <newline/>Recognition Using Convolutional <newline/>Neural Networks

Mao

Dong

Huang

et al. 2014

IEEE Trans. Multimedia

536

198

View full text Add to dashboard Cite

As an essential way of human emotional behavior understanding, speech emotion recognition (SER) has attracted a great deal of attention in human-centered signal processing. Accuracy in SER heavily depends on finding good affect-related, discriminative features. In this paper, we propose to learn affect-salient features for SER using convolutional neural networks (CNN). The training of CNN involves two stages. In the first stage, unlabeled samples are used to learn local invariant features (LIF) using a variant of sparse auto-encoder (SAE) with reconstruction penalization. In the second step, LIF is used as the input to a feature extractor, salient discriminative feature analysis (SDFA), to learn affect-salient, discriminative features using a novel objective function that encourages feature saliency, orthogonality, and discrimination for SER. Our experimental results on benchmark datasets show that our approach leads to stable and robust recognition performance in complex scenes (e.g., with speaker and language variation, and environment distortion) and outperforms several well-established SER features.Index Terms-Affective-salient discriminative feature analysis, convolutional neural networks, feature learning, speech emotion recognition.

show abstract

Section: B Feature Extractionmentioning

confidence: 99%

Learning Salient Features for Speech Emotion <newline/>Recognition Using Convolutional <newline/>Neural Networks

Mao

Dong

Huang

et al. 2014

IEEE Trans. Multimedia

536

198

View full text Add to dashboard Cite

show abstract

“…In previous work on the IEMOCAP database, LLD features are also widely used for acoustic models (e.g., [33][34][35]). However, recent work has shown that knowledge-inspired global prosodic features are more predictive than the LLD features for predicting binary Arousal values [36].…”

Section: Related Workmentioning

confidence: 99%

“…The LSTM-RNN model was used directly for classification in previous work on the IEMOCAP database. Results have shown that LSTM-RNN models have better performance than Hidden Markov Models (e.g., [33,34]). Another application of deep learning methods uses Denoising Autoencoders to model gender information, which is shown to help with the emotion recognition task [37].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Emotion recognition in spontaneous and acted dialogues

Tian

Moore

Lai

2015

2015 International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

In this work, we compare emotion recognition on two types of speech: spontaneous and acted dialogues. Experiments were conducted on the AVEC2012 database of spontaneous dialogues and the IEMOCAP database of acted dialogues. We studied the performance of two types of acoustic features for emotion recognition: knowledge-inspired disfluency and nonverbal vocalisation (DIS-NV) features, and statistical Low-Level Descriptor (LLD) based features. Both Support Vector Machines (SVM) and Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN) were built using each feature set on each emotional database. Our work aims to identify aspects of the data that constrain the effectiveness of models and features. Our results show that the performance of different types of features and models is influenced by the type of dialogue and the amount of training data. Because DIS-NVs are less frequent in acted dialogues than in spontaneous dialogues, the DIS-NV features perform better than the LLD features when recognizing emotions in spontaneous dialogues, but not in acted dialogues. The LSTM-RNN model gives better performance than the SVM model when there is enough training data, but the complex structure of a LSTM-RNN model may limit its performance when there is less training data available, and may also risk over-fitting. Additionally, we find that long distance contexts may be more useful when performing emotion recognition at the word level than at the utterance level.

show abstract

“…2) Features: Previous work on both the AVEC2012 and the IEMOCAP databases have focused on LLD features for the acoustic model (e.g., [25], [26]). However, there are results indicating that knowledge-inspired features, such as global prosodic features, may also be more predictive (e.g., [27], [28]).…”

Section: ) Databasesmentioning

confidence: 99%

Recognizing emotions in dialogues with acoustic and lexical features

Tian

Moore

Lai

2015

2015 International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

Automatic emotion recognition has long been a focus of Affective Computing. We aim at improving the performance of state-of-the-art emotion recognition in dialogues using novel knowledge-inspired features and modality fusion strategies. We propose features based on disfluencies and nonverbal vocalisations (DIS-NVs), and show that they are highly predictive for recognizing emotions in spontaneous dialogues. We also propose the hierarchical fusion strategy as an alternative to current feature-level and decision-level fusion. This fusion strategy combines features from different modalities at different layers in a hierarchical structure. It is expected to overcome limitations of feature-level and decision-level fusion by including knowledge on modality differences, while preserving information of each modality.

show abstract

Analyzing the memory of BLSTM Neural Networks for enhanced emotion classification in dyadic spoken interactions

Cited by 21 publications

References 10 publications

Learning Salient Features for Speech Emotion <newline/>Recognition Using Convolutional <newline/>Neural Networks

Learning Salient Features for Speech Emotion <newline/>Recognition Using Convolutional <newline/>Neural Networks

Emotion recognition in spontaneous and acted dialogues

Recognizing emotions in dialogues with acoustic and lexical features

Contact Info

Product

Resources

About