Modulation spectral features for speech emotion recognition using deep neural networks

Singh, Pahup; Sahidullah, Md; Saha, Goutam

doi:10.1016/j.specom.2022.11.005

Cited by 32 publications

(3 citation statements)

References 93 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human emotions are the key elements in human communication, and speech is fundamental to emotional expression. Speech emotion recognition (SER) aims to distinguish and categorise emotions by extracting and modelling the speech data, such as the short‐term energy, zero‐crossing rate, pitch, formant, prosody, spectral features, temporal features, and Mel frequency cepstral coefficient (MFCC) features [1, 2]. The distributions of the temporal structure, loudness, fundamental frequency, and formant structures are associated with the differences in the emotions communicated in speech.…”

Section: Introductionmentioning

confidence: 99%

DeepCNN: Spectro‐temporal feature representation for speech emotion recognition

Saleem

Gao

Irfan

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

Speech emotion recognition (SER) is an important research problem in human-computer interaction systems. The representation and extraction of features are significant challenges in SER systems. Despite the promising results of recent studies, they generally do not leverage progressive fusion techniques for effective feature representation and increasing receptive fields. To mitigate this problem, this article proposes DeepCNN, which is a fusion of spectral and temporal features of emotional speech by parallelising convolutional neural networks (CNNs) and a convolution layer-based transformer. Two parallel CNNs are applied to extract the spectral features (2D-CNN) and temporal features (1D-CNN) representations. A 2D-convolution layer-based transformer module extracts spectro-temporal features and concatenates them with features from parallel CNNs. The learnt low-level concatenated features are then applied to a deep framework of convolutional blocks, which retrieves high-level feature representation and subsequently categorises the emotional states using an attention gated recurrent unit and classification layer. This fusion technique results in a deeper hierarchical feature representation at a lower computational cost while simultaneously expanding the filter depth and reducing the feature map. The Berlin Database of Emotional Speech (EMO-BD) and Interactive Emotional Dyadic Motion Capture (IEMOCAP) datasets are used in experiments to recognise distinct speech emotions. With efficient spectral and temporal feature representation, the proposed SER model achieves 94.2% accuracy for different emotions on the EMO-BD and 81.1% accuracy on the IEMOCAP dataset respectively. The proposed SER system, DeepCNN, outperforms the baseline SER systems in terms of emotion recognition accuracy on the EMO-BD and IEMOCAP datasets.

show abstract

Section: Introductionmentioning

confidence: 99%

DeepCNN: Spectro‐temporal feature representation for speech emotion recognition

Saleem

Gao

Irfan

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

show abstract

“…Singh et al [6] proposed an approach for speech emotion recognition using a deep neural network with support vector machine (DNN-SVM) model. The proposed model employed the constant-Q transform based modulation spectral features (CQT-MSF) to process the speech data.…”

Section: A Traditional Machine Learningmentioning

confidence: 99%

SCQT-MaxViT: Speech Emotion Recognition With Constant-Q Transform and Multi-Axis Vision Transformer

Ong,

Lee,

Lim

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Speech emotion recognition presents a significant challenge within the field of affective computing, requiring the analysis and detection of emotions conveyed through speech signals. However, existing approaches often rely on traditional signal processing techniques and handcrafted features, which may not effectively capture the nuanced aspects of emotional expression. In this paper, an approach named ''SCQT-MaxViT'' is proposed for speech emotion recognition, combining signal processing, computer vision, and deep learning techniques. The method utilizes the Constant-Q Transform (CQT) to convert speech waveforms into spectrograms, providing high-frequency resolution and enabling the model to capture intricate emotional details. Additionally, the Multi-axis Vision Transformer (MaxViT) is employed for further representation learning and classification of the CQT spectrograms. MaxViT incorporates a multi-axis self-attention mechanism, facilitating both local and global interactions within the network and enhancing the ability of the model to learn meaningful features. Furthermore, the dataset is augmented using random time masking techniques to enhance the generalization capabilities. Achieving accuracies of 88.68% on the Emo-DB dataset, 77.54% on the RAVDESS dataset, and 62.49% on the IEMOCAP dataset, the proposed SCQT-MaxViT method exhibits promising performance in capturing and recognizing emotions in speech signals.

show abstract

“…Many voice features have been discovered by the academicians and other researchers. Some of these include formant frequency [22]. Formant frequencies are a sort of resonance frequencies.…”

Section: Introductionmentioning

confidence: 99%

A Novel Voice Feature AVA and its Application to the Pathological Voice Detection Through Machine Learning

Altaf,

Mahdin,

Maskat

et al. 2023

IJACSA

View full text Add to dashboard Cite

Voice pathology is a universal problem which must be addressed. Traditionally, this malady is treated by using the surgical instruments in the varied healthcare settings. In the current era, machine learning experts have paid an increasing attention towards the solution of this problem by exploiting the signal processing of the voice. For this purpose, numerous voice features have been capitalized to classify the healthy and pathological voice signals. In particular, Mel-Frequency Cepstral Coefficients (MFCC) is a widely used feature in speech and audio signal processing. It denotes spectral characteristics of a voice signal, particularly of human speech. The modus operandi of MFCC is too time-consuming, which goes against the hasty and urgent nature of the modern times. This study has developed a yet another voice feature by utilizing the average value of the amplitudes (AVA) of the voice signals. Moreover, Gaussian Naive Bayes classifier has been employed to classify the given voice signals as healthy or pathological. Apart from that, the dataset has been acquired from the SVD (Saarbrucken Voice Database) to demonstrate the workability of the proposed voice feature and its usage in the classifier. The machine experimentation rendered very promising results. Particularly, Recall, F1 and accuracy scores obtained, are 100%, 83% and 80%, respectively. These results vividly imply that the proposed classifier can be installed in various healthcare settings.

show abstract

Modulation spectral features for speech emotion recognition using deep neural networks

Cited by 32 publications

References 93 publications

DeepCNN: Spectro‐temporal feature representation for speech emotion recognition

DeepCNN: Spectro‐temporal feature representation for speech emotion recognition

SCQT-MaxViT: Speech Emotion Recognition With Constant-Q Transform and Multi-Axis Vision Transformer

A Novel Voice Feature AVA and its Application to the Pathological Voice Detection Through Machine Learning

Contact Info

Product

Resources

About