Soft-Target Training with Ambiguous Emotional Utterances for DNN-Based Speech Emotion Classification

Ando, Atsushi; Kobashikawa, Satoshi; Kamiyama, Hosana; Masumura, Ryo; Ijima, Yusuke; Aono, Yushi

doi:10.1109/icassp.2018.8461299

Cited by 41 publications

(27 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though listener variability affects the majority decision as to emotion perception, there is little work that considers the listener in SER. One related work is soft-label / multi-label emotion recognition; it models the distribution of emotion perception of listeners [17,29,30], but it cannot distinguish individuals. In music emotion recognition, several studies have tackled listener-wise perception [31,32].…”

Section: I R E L a T E D W O R Kmentioning

confidence: 99%

“…The database consists of a total of 12 h of English utterances generated by improvised or scripted scenarios specifically atsushi ando et al written to represent the emotional expressions. As in several conventional studies [11,14,16,27,29], we used only audio tracks of the improvised set since scripted data may contain undesired contextual information. There are six listeners in the corpus and every utterance was annotated by three of them.…”

Section: A) Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Speech emotion recognition based on listener-dependent emotion perception models

Ando

Mori

Kobashikawa

et al. 2021

SIP

Self Cite

View full text Add to dashboard Cite

This paper presents a novel speech emotion recognition scheme that leverages the individuality of emotion perception. Most conventional methods simply poll multiple listeners and directly model the majority decision as the perceived emotion. However, emotion perception varies with the listener, which forces the conventional methods with their single models to create complex mixtures of emotion perception criteria. In order to mitigate this problem, we propose a majority-voted emotion recognition framework that constructs listener-dependent (LD) emotion recognition models. The LD model can estimate not only listener-wise perceived emotion, but also majority decision by averaging the outputs of the multiple LD models. Three LD models, fine-tuning, auxiliary input, and sub-layer weighting, are introduced, all of which are inspired by successful domain-adaptation frameworks in various speech processing tasks. Experiments on two emotional speech datasets demonstrate that the proposed approach outperforms the conventional emotion recognition frameworks in not only majority-voted but also listener-wise perceived emotion recognition.

show abstract

Section: I R E L a T E D W O R Kmentioning

confidence: 99%

Section: A) Datasetsmentioning

confidence: 99%

Speech emotion recognition based on listener-dependent emotion perception models

Ando

Mori

Kobashikawa

et al. 2021

SIP

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are several studies that leverage listener-dependent emotion perceptions for SER. One approach is to model the distribution of emotion perception of listeners by softlabel [14] or multivariate Gaussian [15]. However, it cannot make distinctions with regard to the listeners.…”

Section: Introductionmentioning

confidence: 99%

Speech Emotion Recognition Based on Listener Adaptive Models

Ando

Masumura

Sato

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

This paper presents a novel speech emotion recognition scheme that can deal with the individuality of emotion perception. Most conventional methods directly model the majority decision of multiple listener's perceived emotions. However, emotion perception varies with the listener, which means the conventional methods can mismatch the recognition results to human perception. In order to mitigate this problem, we propose a Listener Adaptive (LA) model that reflects emotion recognition criteria of each listener. One-hot listener codes with several adaptation layers are employed in the LA model. The LA model yields the posterior probabilities of the listener-specific perceived emotions. Majority-voted emotion can be also estimated by averaging, in the LA model, the posterior probabilities for all listeners. Experiments on two emotional speech datasets demonstrate that the proposed approach offers improved listener-wise perceived emotion recognition performance in natural speech.

show abstract

“…Specially, the human brain's attention to each area of an image is different, and it will thus focus on certain areas with obvious characteristics. Recently, the attention mechanism has been successfully applied for object detection [30,31] and classification [32], nature language processing (NLP) [33,34], machine translation [35,36], etc. In order to extract more discriminative deep speech features for improving recognition performance, some attention-based deep models have been developed for SER in recent years.…”

Section: Introductionmentioning

confidence: 99%

Speech Emotion Recognition by Combining a Unified First-Order Attention Network With Data Balance

et al. 2020

View full text Add to dashboard Cite

Soft-Target Training with Ambiguous Emotional Utterances for DNN-Based Speech Emotion Classification

Cited by 41 publications

References 6 publications

Speech emotion recognition based on listener-dependent emotion perception models

Speech emotion recognition based on listener-dependent emotion perception models

Speech Emotion Recognition Based on Listener Adaptive Models

Speech Emotion Recognition by Combining a Unified First-Order Attention Network With Data Balance

Contact Info

Product

Resources

About