A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition

Abstract

Speech is the most significant mode of communication among human beings and a potential method for human-computer interaction (HCI) by using a microphone sensor. Quantifiable emotion recognition using these sensors from speech signals is an emerging area of research in HCI, which applies to multiple applications such as human-reboot interaction, virtual reality, behavior assessment, healthcare, and emergency call centers to determine the speaker's emotional state from an individual's speech. In this paper, we present major contributions for; (i) increasing the accuracy of speech emotion recognition (SER) compared to state of the art and (ii) reducing the computational complexity of the presented SER model. We propose an artificial intelligence-assisted deep stride convolutional neural network (DSCNN) architecture using the plain nets strategy to learn salient and discriminative features from spectrogram of speech signals that are enhanced in prior steps to perform better. Local hidden patterns are learned in convolutional layers with special strides to down-sample the feature maps rather than pooling layer and global discriminative features are learned in fully connected layers. A SoftMax classifier is used for the classification of emotions in speech. The proposed technique is evaluated on Interactive Emotional Dyadic Motion Capture (IEMOCAP) and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) datasets to improve accuracy by 7.85% and 4.5%, respectively, with the model size reduced by 34.5 MB. It proves the effectiveness and significance of the proposed SER technique and reveals its applicability in real-world applications.

Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition

“…Mustaqeem and Kwon [ 33 ] revealed the amount of energy transmitted by a sound wave is correlated to the amplitude of a sound wave. The amplitude of a sound wave denotes the maximum displacement of an element in the middle from its rest location.…”

“…This method has a drawback in that the classifying emotions can be time consuming because the audio file must be analyzed and converted to audio without noise or silence for preprocessing. In the aforementioned studies [ 29 , 30 , 31 , 32 , 33 ], local correlations between spectral features could be ignored by using normalized spectral features from pre-processing.…”

A Speech Emotion Recognition Model Based on Multi-Level Local Binary and Local Ternary Patterns

“…Local features are spectral lowlevel descriptors (LLDs) (Mel Filter Bank, MFCCs), energy and voice LLDs (loudness, F0, jitter, shimmer); global features involve functions extracted from the LLDs, such as maximum, minimum, mean, standard deviation, duration, regression coefficients [20]. The deep learning methods are deep neural networks (DNN), deep stacked auto-encoder (SAE), convolutional neural network (CNN) [21], [22], longshort term memory network (LSTM) [21], recurrent neural network (RNN) and other similar methods [23].…”

“…Multi-layer perceptron is also a supervised neural network learning model used in the SER [35]. Deep-learning classifiers widely used in the SER domain as classification techniques are RNN [23], CNN [21], [22], DNN [36], LSTM network [8], auto encoders, multitask learning, transfer learning, and attention mechanism [6], [15].…”

Clustering-Based Speech Emotion Recognition by Incorporating Learned Features and Deep BiLSTM

“…Nowadays, mostly researchers utilize deep learning techniques for SER using Mel-scale filter bank speech spectrogram as an input feature. A spectrogram is a 2-D representation of speech signals which is widely used in convolutional neural networks (CNNs) for extracting the salient and discriminative features in SER [2] and other signal processing applications [3], [4]. Mostly 2-D CNNs are specially designed for visual recognition tasks [5]- [7] and researchers are inspired by their performance to explore 2-D CNNs in the field of SER.…”