Speech Emotion Recognition with Dual-Sequence LSTM Architecture

Wang, Jianyou; Xue, Michael; Culhane, Ryan; Diao, Enmao; Ding, Jie; Tarokh, Vahid

doi:10.1109/icassp40776.2020.9054629

Cited by 109 publications

(54 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors employ Low-Level Descriptors (LLDs) extracted from the audio signal as input. (Wang et al, 2020) propose a model consisting of two jointly trained LSTMs: each of these models is separately used to process MFCC features and Mel-spectrograms. Both models predict an output class (emotion) that is averaged to arrive at the result.…”

Section: Related Workmentioning

confidence: 99%

A Study on the Ambiguity in Human Annotation of German Oral History Interviews for Perceived Emotion Recognition and Sentiment Analysis

Gref¹,

Matthiesen²,

Venugopala³

et al. 2022

Preprint

View full text Add to dashboard Cite

For research in audiovisual interview archives often it is not only of interest what is said but also how. Sentiment analysis and emotion recognition can help capture, categorize and make these different facets searchable. In particular, for oral history archives, such indexing technologies can be of great interest. These technologies can help understand the role of emotions in historical remembering. However, humans often perceive sentiments and emotions ambiguously and subjectively. Moreover, oral history interviews have multi-layered levels of complex, sometimes contradictory, sometimes very subtle facets of emotions. Therefore, the question arises of the chance machines and humans have capturing and assigning these into predefined categories. This paper investigates the ambiguity in human perception of emotions and sentiment in German oral history interviews and the impact on machine learning systems. Our experiments reveal substantial differences in human perception for different emotions. Furthermore, we report from ongoing machine learning experiments with different modalities. We show that the human perceptual ambiguity and other challenges, such as class imbalance and lack of training data, currently limit the opportunities of these technologies for oral history archives. Nonetheless, our work uncovers promising observations and possibilities for further research.

show abstract

Section: Related Workmentioning

confidence: 99%

A Study on the Ambiguity in Human Annotation of German Oral History Interviews for Perceived Emotion Recognition and Sentiment Analysis

Gref¹,

Matthiesen²,

Venugopala³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We used IEMOCAP dataset [29], a benchmark dataset containing 12 hours of speech from 10 professional actors. Following the literature [30,31,32,33], we extracted 5531 utterances of four emotion types from the dataset: 1636 happy (also including excited), 1084 sad, 1103 angry, and 1708 neutral. The utterances were forced aligned using the P2FA forced aligner.…”

Section: Datamentioning

confidence: 99%

Pause-Encoded Language Models for Recognition of Alzheimer’s Disease and Emotion

Yuan¹,

Cai²,

Church³

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We propose enhancing Transformer language models (BERT, RoBERTa) to take advantage of pauses. Pauses play an important role in speech. In previous work we developed a method to encode pauses in transcripts for recognition of Alzheimer's disease. In this study, we extend this idea to language models. We re-train BERT and RoBERTa using a large collection of pause-encoded transcripts, and conduct finetuning for two downstream tasks, recognition of Alzheimer's disease and emotion. Pause-encoded language models outperform text-only language models on these tasks. Pause augmentation by duration perturbation for training is shown to improve pause-encoded language models.

show abstract

“…As phishing URLs comprise sequences of characters and words, existing CNN and LSTM networks are known as the amenable method for URL feature extraction. The combination of the CNN and LSTM for estimating the time-invariant filter coefficients is already widely used in the field of spatiotemporal feature modeling [13], [14].…”

Section: Triplet Network For Embedding Of Phishing Attacksmentioning

confidence: 99%

Integrating Deep Learning with First-Order Logic Programmed Constraints for Zero-Day Phishing Attack Detection

Cho

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Considering the fatality of phishing attacks that are emphasized by many organizations, the inductive learning approach using reported malicious URLs has been verified in the field of deep learning. However, the deep learning-based method mainly focused on the fitting of a classification task via historical URL observation shows a limitation of recall due to the characteristics of zero-day attack. In order to model the nature of a zero-day phishing attack in which URL addresses are generated and discarded immediately, an approach that utilizes the expert knowledge is promising. We introduce the integration method of deep learning and logic programmed domain knowledge to inject the real-world constraints. We design neural and logic classifiers and propose the joint learning method of each component based on the traditional neuro-symbolic integration. Extensive experiments on three real-world datasets consisting of 222,541 URLs showed the highest recall among the latest deep learning methods, despite the hostile class-imbalanced condition. We demonstrate that the optimized weighting between neural and logic component has an effect of improving the recall over 3% compared to the existing methods.

show abstract

Speech Emotion Recognition with Dual-Sequence LSTM Architecture

Cited by 109 publications

References 16 publications

A Study on the Ambiguity in Human Annotation of German Oral History Interviews for Perceived Emotion Recognition and Sentiment Analysis

A Study on the Ambiguity in Human Annotation of German Oral History Interviews for Perceived Emotion Recognition and Sentiment Analysis

Pause-Encoded Language Models for Recognition of Alzheimer’s Disease and Emotion

Integrating Deep Learning with First-Order Logic Programmed Constraints for Zero-Day Phishing Attack Detection

Contact Info

Product

Resources

About