An Investigation of a Feature-Level Fusion for Noisy Speech Emotion Recognition

Sekkate, Sara; Adib, Abdellah; Jebara, Sofia Ben

doi:10.3390/computers8040091

Cited by 19 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This conclusion has been further confirmed in [35]. Additionally, the method strongly relies on prior knowledge of noise [36], which limits their applications. Hsu et al [37] explored adversarial training for disentangling speaker attribute from noise attribute.…”

Section: Introductionmentioning

confidence: 60%

Noise-robust voice conversion with domain adversarial training

Du¹,

Xie²,

Li³

2022

Preprint

View full text Add to dashboard Cite

Voice conversion has made great progress in the past few years under the studio-quality test scenario in terms of speech quality and speaker similarity. However, in real applications, test speech from source speaker or target speaker can be corrupted by various environment noises, which seriously degrade the speech quality and speaker similarity. In this paper, we propose a novel encoderdecoder based noise-robust voice conversion framework, which consists of a speaker encoder, a content encoder, a decoder, and two domain adversarial neural networks. Specifically, we integrate disentangling speaker and content representation technique with domain adversarial training technique. Domain adversarial training makes speaker representations and content representations extracted by speaker encoder and content encoder from clean speech and noisy speech in the same space, respectively. In this way, the learned speaker and content representations are noise-invariant. Therefore, the two noise-invariant representations can be taken as input by the decoder to predict the clean converted spectrum. The experimental results demonstrate that our proposed method can synthesize clean converted speech under noisy test scenarios, where the source speech and target speech can be corrupted by seen or unseen noise types during the training process. Additionally, both speech quality and speaker similarity are improved.

show abstract

Section: Introductionmentioning

confidence: 60%

Noise-robust voice conversion with domain adversarial training

Du¹,

Xie²,

Li³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Wang et al [23] took advantage of the sequential floating forward search (SFFS) method to select the most efficient feature subset of wavelet packets decomposition based on the RBF kernel support vector machine for speaker-independent. Sekkate et al [24] employed a relatively low-dimensional feature set that combines three features including mel frequency cepstral coefficients (MFCCs) derived from discrete wavelet transform (DWT) sub-band coefficients (DMFCC), and they utilized feature extraction algorithm from above feature set to acquire better results based on the clean conditions and several noised environments in specific emotional databases. Finally, we summarize above the aforementioned works shown in Table 1, where SI and SD are speaker-independent and speaker-dependent respectively.…”

Section: Related Workmentioning

confidence: 99%

“…The feature set includes different kinds of speech features such as: energy, pitch, voiced, formant and cepstrum. For example, pitch, also known as fundamental frequency F0, is a feature that corresponds to the frequency of vibration of the vocal folds [24].…”

Section: Wavelet Packet Reconstruction Sequence Generationmentioning

confidence: 99%

Bulletin of the Polish Academy of Sciences: Technical Sciences

Meng,

Yan,

Wei

et al. 2021

View full text Add to dashboard Cite

Speech emotion recognition (SER) is a complicated and challenging task in the human-computer interaction because it is difficult to find the best feature set to discriminate the emotional state entirely. We always used the FFT to handle the raw signal in the process of extracting the low-level description features, such as short-time energy, fundamental frequency, formant, MFCC (mel frequency cepstral coefficient) and so on. However, these features are built on the domain of frequency and ignore the information from temporal domain. In this paper, we propose a novel framework that utilizes multi-layers wavelet sequence set from wavelet packet reconstruction (WPR) and conventional feature set to constitute mixed feature set for achieving the emotional recognition with recurrent neural networks (RNN) based on the attention mechanism. In addition, the silent frames have a disadvantageous effect on SER, so we adopt voice activity detection of autocorrelation function to eliminate the emotional irrelevant frames. We show that the application of proposed algorithm significantly outperforms traditional features set in the prediction of spontaneous emotional states on the IEMOCAP corpus and EMODB database respectively, and we achieve better classification for both speaker-independent and speaker-dependent experiment. It is noteworthy that we acquire 62.52% and 77.57% accuracy results with speaker-independent (SI) performance, 66.90% and 82.26% accuracy results with speaker-dependent (SD) experiment in final.

show abstract

A Statistical Based Modeling Approach for Deep Learning Based Speech Emotion Recognition

Sekkate

Adib

2021

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

An Investigation of a Feature-Level Fusion for Noisy Speech Emotion Recognition

Cited by 19 publications

References 42 publications

Noise-robust voice conversion with domain adversarial training

Noise-robust voice conversion with domain adversarial training

Bulletin of the Polish Academy of Sciences: Technical Sciences

A Statistical Based Modeling Approach for Deep Learning Based Speech Emotion Recognition

Contact Info

Product

Resources

About