Recognizing emotions in dialogues with acoustic and lexical features

Tian, Leimin; Moore, Johanna D.; Lai, Catherine

doi:10.1109/acii.2015.7344651

Cited by 6 publications

(6 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The feature values are the ratios between the durations of DIS-NV events and the utterance duration. Compared to the AVEC2012 database of spontaneous dialogue, DIS-NVs are less frequent in the IEMOCAP database of acted dialogue, which limits their performance on emotion recognition in acted dialogue [21,44].…”

Section: Disfluency and Non-verbal Vocalisation (Dis-nv) Featuresmentioning

confidence: 99%

Recognizing emotions in spoken dialogue with hierarchically fused acoustic and lexical features

Tian

Moore

Lai

2016

2016 IEEE Spoken Language Technology Workshop (SLT)

Self Cite

View full text Add to dashboard Cite

Automatic emotion recognition is vital for building natural and engaging human-computer interaction systems. Combining information from multiple modalities typically improves emotion recognition performance. In previous work, features from different modalities have generally been fused at the same level with two types of fusion strategies: Feature-Level fusion, which concatenates feature sets before recognition; and Decision-Level fusion, which makes the final decision based on outputs of the unimodal models. However, different features may describe data at different time scales or have different levels of abstraction. Cognitive Science research also indicates that when perceiving emotions, humans use information from different modalities at different cognitive levels and time steps. Therefore, we propose a Hierarchical fusion strategy for multimodal emotion recognition, which incorporates global or more abstract features at higher levels of its knowledge-inspired structure.We build multimodal emotion recognition models combining state-of-the-art acoustic and lexical features to study the performance of the proposed Hierarchical fusion. Experiments on two emotion databases of spoken dialogue show that this fusion strategy consistently outperforms both Feature-Level and Decision-Level fusion. The multimodal emotion recognition models using the Hierarchical fusion strategy achieved state-of-the-art performance on recognizing emotions in both spontaneous and acted dialogue.

show abstract

Section: Disfluency and Non-verbal Vocalisation (Dis-nv) Featuresmentioning

confidence: 99%

Recognizing emotions in spoken dialogue with hierarchically fused acoustic and lexical features

Tian

Moore

Lai

2016

2016 IEEE Spoken Language Technology Workshop (SLT)

Self Cite

View full text Add to dashboard Cite

show abstract

“…and audio features 1 { } n q i i y R = ∈ , we could obtain the transformed features for arbitrary visual or audio feature by Equation (27). Based on these transformed features, we used feature-level and decision-level fusion strategies, respectively, to evaluate the effectiveness of the ICDKCFA for continuous dimensional emotion recognition.…”

Section: Methodsmentioning

confidence: 99%

“…Sayedelahl et al [20] presented a combined bi-modal feature-decision fusion approach to improve the performance of emotion recognition. Tian et al [27] proposed a hierarchical fusion strategy to combine features from different modalities at different layers of a hierarchical structure.…”

Section: Introductionmentioning

confidence: 99%

Incomplete Cholesky Decomposition based Kernel Cross Modal Factor Analysis for Audiovisual Continuous Dimensional Emotion Recognition

2019

KSII TIIS

View full text Add to dashboard Cite

Recently, continuous dimensional emotion recognition from audiovisual clues has attracted increasing attention in both theory and in practice. The large amount of data involved in the recognition processing decreases the efficiency of most bimodal information fusion algorithms. A novel algorithm, namely the incomplete Cholesky decomposition based kernel cross factor analysis (ICDKCFA), is presented and employed for continuous dimensional audiovisual emotion recognition, in this paper. After the ICDKCFA feature transformation, two basic fusion strategies, namely feature-level fusion and decision-level fusion, are explored to combine the transformed visual and audio features for emotion recognition. Finally, extensive experiments are conducted to evaluate the ICDKCFA approach on the AVEC 2016 Multimodal Affect Recognition Sub-Challenge dataset. The experimental results show that the ICDKCFA method has a higher speed than the original kernel cross factor analysis with the comparable performance. Moreover, the ICDKCFA method achieves a better performance than other common information fusion methods, such as the Canonical correlation analysis, kernel canonical correlation analysis and cross-modal factor analysis based fusion methods.

show abstract

“…Mohamed R.Amer et al [48] used the deep networks technique and designed a model for speech emotion detection. Leimen Tian et al [49] used the LSTM technique and developed a model for recognizing emotions in dialogues with acoustic and lexical features. The authors concluded that this model had the potential to improve the quality of emotional interactions in current dialogue systems.…”

Section: Survey On Other Deep Learning Techniquesmentioning

confidence: 99%

A Review on Deep Learning Algorithms for Speech and Facial Emotion Recognition

Latha

Priya

2016

APTIKOM Journal Comput. Sci. Inf. Technol.

View full text Add to dashboard Cite

Deep Learning is the recent machine learning technique that tries to model high level abstractions in data by using multiple processing layers with complex structures. It is also known as deep structured learning, hierarchical learning or deep machine learning. The term “deep learning" indicates the method used in training multi-layered neural networks. Deep Learning technique has obtained remarkable success in the field of face recognition with 97.5% accuracy. Facial Electromyogram (FEMG) signals are used to detect the different emotions of humans. Some of the deep learning techniques discussed in this paper are Deep Boltzmann Machine (DBM), Deep Belief Networks (DBN), Convolutional Neural Networks (CNN) and Stacked Auto Encoders respectively. This paper focuses on the review of some of the deep learning techniques used by various researchers which paved the way to improve the classification accuracy of the FEMG signals as well as the speech signals.

show abstract

Recognizing emotions in dialogues with acoustic and lexical features

Cited by 6 publications

References 39 publications

Recognizing emotions in spoken dialogue with hierarchically fused acoustic and lexical features

Recognizing emotions in spoken dialogue with hierarchically fused acoustic and lexical features

Incomplete Cholesky Decomposition based Kernel Cross Modal Factor Analysis for Audiovisual Continuous Dimensional Emotion Recognition

A Review on Deep Learning Algorithms for Speech and Facial Emotion Recognition

Contact Info

Product

Resources

About