Cascaded Convolutional Neural Network Architecture for Speech Emotion Recognition in Noisy Conditions

Nam, Youngja; Lee, Chankyu

doi:10.3390/s21134399

Cited by 27 publications

(11 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Latif et al [21] also employed several DL models on raw speech. Several modified DL models are also observed for SER, including capsule neural network [22], 3D CNN-LSTM model [10], 3D CNN using k-means clustering [8], 2D CNN with a self-attention dilated residual network [9], Spiking Neural Network (SNN) [23], convolutional capsule (Conv-Cap) and bi-directional gated recurrent unit (Bi-GRU) [24], attention-LSTM-attention [25], Bi-GRU with attention mechanism [26], CNN with a capsule neural network (Caps Net) [13], temporal CNN with self-attention transfer network (SATN) [27], 1D CNN based on the multi-learning trick (MLT) [14], cascaded denoising CNN (Dn-CNN) [28], and a pre-trained deep CNN model with attention [29]. The learning features are the main attraction of different DL-based SER methods.…”

Section: Ser With Deep Learning (Dl) and Signal Transformationmentioning

confidence: 99%

Recognition of Emotion with Intensity from Speech Signal Using 3D Transformed Feature and Deep Learning

et al. 2022

View full text Add to dashboard Cite

Speech Emotion Recognition (SER), the extraction of emotional features with the appropriate classification from speech signals, has recently received attention for its emerging social applications. Emotional intensity (e.g., Normal, Strong) for a particular emotional expression (e.g., Sad, Angry) has a crucial influence on social activities. A person with intense sadness or anger may fall into severe disruptive action, eventually triggering a suicidal or devastating act. However, existing Deep Learning (DL)-based SER models only consider the categorization of emotion, ignoring the respective emotional intensity, despite its utmost importance. In this study, a novel scheme for Recognition of Emotion with Intensity from Speech (REIS) is developed using the DL model by integrating three speech signal transformation methods, namely Mel-frequency Cepstral Coefficient (MFCC), Short-time Fourier Transform (STFT), and Chroma STFT. The integrated 3D form of transformed features from three individual methods is fed into the DL model. Moreover, under the proposed REIS, both the single and cascaded frameworks with DL models are investigated. A DL model consists of a 3D Convolutional Neural Network (CNN), Time Distribution Flatten (TDF) layer, and Bidirectional Long Short-term Memory (Bi-LSTM) network. The 3D CNN block extracts convolved features from 3D transformed speech features. The convolved features were flattened through the TDF layer and fed into Bi-LSTM to classify emotion with intensity in a single DL framework. The 3D transformed feature is first classified into emotion categories in the cascaded DL framework using a DL model. Then, using a different DL model, the intensity level of the identified categories is determined. The proposed REIS has been evaluated on the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) benchmark dataset, and the cascaded DL framework is found to be better than the single DL framework. The proposed REIS method has shown remarkable recognition accuracy, outperforming related existing methods.

show abstract

Section: Ser With Deep Learning (Dl) and Signal Transformationmentioning

confidence: 99%

Recognition of Emotion with Intensity from Speech Signal Using 3D Transformed Feature and Deep Learning

et al. 2022

View full text Add to dashboard Cite

show abstract

“…With respect to the recognition of speech and birdsong, many researchers often learn multi-scale features directly from waveforms 42 – 44 or use short-time Fourier transforms 19 , 45 , and Mel filters 21 , 46 to generate spectrograms as input into CNN. Mel filtering is designed to imitate human hearing habits, and there is a lack of evidence about whether birds have the same characteristics.…”

Section: Resultsmentioning

confidence: 99%

Birdsong classification based on ensemble multi-scale convolutional neural network

Liu

Zhang

et al. 2022

Sci Rep

View full text Add to dashboard Cite

With the intensification of ecosystem damage, birds have become the symbolic species of the ecosystem. Ornithology with interdisciplinary technical research plays a great significance for protecting birds and evaluating ecosystem quality. Deep learning shows great progress for birdsongs recognition. However, as the number of network layers increases in traditional CNN, semantic information gradually becomes richer and detailed information disappears. Secondly, the global information carried by the entire input may be lost in convolution, pooling, or other operations, and these problems will weaken the performance of classification. In order to solve such problems, based on the feature spectrogram from the wavelet transform for the birdsongs, this paper explored the multi-scale convolution neural network (MSCNN) and proposed an ensemble multi-scale convolution neural network (EMSCNN) classification framework. The experiments compared the MSCNN and EMSCNN models with other CNN models including LeNet, VGG16, ResNet101, MobileNetV2, EfficientNetB7, Darknet53 and SPP-net. The results showed that the MSCNN model achieved an accuracy of 89.61%, and EMSCNN achieved an accuracy of 91.49%. In the experiments on the recognition of 30 species of birds, our models effectively improved the classification effect with high stability and efficiency, indicating that the models have better generalization ability and are suitable for birdsongs species recognition. It provides methodological and technical scheme reference for bird classification research.

show abstract

“…Ren et al [34] used the two-stage cascaded model to select the most promising candidate sets to achieve the image set classification task. Nam et al [35] removed noise signal in the first stage and realize target signal classification in the second stage.…”

Section: Methodsmentioning

confidence: 99%

Automated lung ultrasound scoring for evaluation of coronavirus disease 2019 pneumonia using two-stage cascaded deep learning model

Xing

et al. 2022

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

Coronavirus disease 2019 (COVID-19) pneumonia has erupted worldwide, causing massive population deaths and huge economic losses. In clinic, lung ultrasound (LUS) plays an important role in the auxiliary diagnosis of COVID-19 pneumonia. However, the lack of medical resources leads to the low using efficiency of the LUS, to address this problem, a novel automated LUS scoring system for evaluating COVID-19 pneumonia based on the two-stage cascaded deep learning model was proposed in this paper. 18,330 LUS images collected from 26 COVID-19 pneumonia patients were successfully assigned scores by two experienced doctors according to the designed four-level scoring standard for training the model. At the first stage, we made a secondary selection of these scored images through five ResNet-50 models and five-fold cross validation to obtain the available 12,949 LUS images which were highly relevant to the initial scoring results. At the second stage, three deep learning models including ResNet-50, Vgg-19, and GoogLeNet were formed the cascaded scored model and trained using the new dataset, whose predictive result was obtained by the voting mechanism. In addition, 1000 LUS images collected another 5 COVID-19 pneumonia patients were employed to test the model. Experiments results showed that the automated LUS scoring model was evaluated in terms of accuracy, sensitivity, specificity, and F1-score, being 96.1%, 96.3%, 98.8%, and 96.1%, respectively. They proved the proposed two-stage cascaded deep learning model could automatically score an LUS image, which has great potential for application to the clinics on various occasions.

show abstract

Cascaded Convolutional Neural Network Architecture for Speech Emotion Recognition in Noisy Conditions

Cited by 27 publications

References 45 publications

Recognition of Emotion with Intensity from Speech Signal Using 3D Transformed Feature and Deep Learning

Recognition of Emotion with Intensity from Speech Signal Using 3D Transformed Feature and Deep Learning

Birdsong classification based on ensemble multi-scale convolutional neural network

Automated lung ultrasound scoring for evaluation of coronavirus disease 2019 pneumonia using two-stage cascaded deep learning model

Contact Info

Product

Resources

About