GM-TCNet: Gated Multi-scale Temporal Convolutional Network using Emotion Causality for Speech Emotion Recognition

Ye, Jia-Xin; Xin-Cheng, Wen,; Wang, Xuan-Ze; Xu, Yong; Luo, Yan; Wu, Changli; Chen, Liyan; Liu, Kun-Hong

doi:10.1016/j.specom.2022.07.005

Cited by 32 publications

(7 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To solve the problem that the Deep Autoencoding Gaussian Mixture Model (DAGMM) is defective in preserving the input topology, Chen et al [26] proposed a Self-Organizing Map-assisted depth Autoencoding Gaussian Mixture Model (SOM-DAGMM), which overcomes the above drawbacks of DAGMM by well balancing the low dimensionality requirement and topology preservation requirement of Gaussian Mixture Model (GMM). In addition, Ye et al [27] explored how to model sequence information in dynamic time scales for learning multi-scale contextual sentiment representations at different scales. This method provides new ideas for network traffic feature representation in industrial Internet.…”

Section: Supervised Learning-based Intrusion Detectionmentioning

confidence: 99%

“…For two identical sub-blocks in the j th MRBlock, the Dilated Causal Convolution in each MRBlock starts with a dilation rate of 2 j−1 . The exponentially growing dilation rate allows a rapid increase in the receptive field to capture a more extended range of dependencies [27]. Also, the causal constraints ensure that the model does not leak future information.We use the add operation and the global average pooling operation to fuse forward and backward complementary contextual information about network traffic, as implemented below:…”

Section: Improved Multi-scale Temporal Convolutional Networkmentioning

confidence: 99%

See 1 more Smart Citation

An Industrial Intrusion Detection Method Based on Hybrid Convolutional Neural Networks with Improved TCN

Liu

2024

CMC

View full text Add to dashboard Cite

Network intrusion detection systems (NIDS) based on deep learning have continued to make significant advances. However, the following challenges remain: on the one hand, simply applying only Temporal Convolutional Networks (TCNs) can lead to models that ignore the impact of network traffic features at different scales on the detection performance. On the other hand, some intrusion detection methods consider multi-scale information of traffic data, but considering only forward network traffic information can lead to deficiencies in capturing multiscale temporal features. To address both of these issues, we propose a hybrid Convolutional Neural Network that supports a multi-output strategy (BONUS) for industrial internet intrusion detection. First, we create a multiscale Temporal Convolutional Network by stacking TCN of different scales to capture the multiscale information of network traffic. Meanwhile, we propose a bi-directional structure and dynamically set the weights to fuse the forward and backward contextual information of network traffic at each scale to enhance the model's performance in capturing the multi-scale temporal features of network traffic. In addition, we introduce a gated network for each of the two branches in the proposed method to assist the model in learning the feature representation of each branch. Extensive experiments reveal the effectiveness of the proposed approach on two publicly available traffic intrusion detection datasets named UNSW-NB15 and NSL-KDD with F1 score of 85.03% and 99.31%, respectively, which also validates the effectiveness of enhancing the model's ability to capture multi-scale temporal features of traffic data on detection performance.

show abstract

Section: Supervised Learning-based Intrusion Detectionmentioning

confidence: 99%

Section: Improved Multi-scale Temporal Convolutional Networkmentioning

confidence: 99%

An Industrial Intrusion Detection Method Based on Hybrid Convolutional Neural Networks with Improved TCN

Liu

2024

CMC

View full text Add to dashboard Cite

show abstract

“…According to the Hoehn and Yahr scale, speech data in the MDVR-KCL dataset are classifed into four categories: healthy individuals, PD1 level, PD2 level, and PD3 level, which is completely marked by expert evaluation scores. We compared fve deep learning methods [48][49][50][51], including models based on convolutional neural networks, transformers, and transfer learning for speech emotion recognition, and achieved good results, which is sufcient to prove that these deep models can evaluate the severity of Parkinson's disease speech, and the efectiveness of the proposed TmmNet is also remarkable.…”

Section: Global Evaluationmentioning

confidence: 98%

A Triplet Multimodel Transfer Learning Network for Speech Disorder Screening of Parkinson’s Disease

Zhao,

Wang,

Niu

et al. 2024

International Journal of Intelligent Systems

View full text Add to dashboard Cite

Deterioration in the quality of a person’s voice and speech is an early sign of Parkinson’s disease (PD). Although a number of computer-based methods have been invested to use patients’ speech for early diagnosis of Parkinson’s disease, they only focus on a fixed pronunciation test, such as the subjects’ monosyllabic pronunciation is analyzed to determine whether they have potential possibility of PD. Moreover, only using traditional speech analysis methods to extract single-view speech features cannot provide a comprehensive feature representation. This paper is dedicated to the study of various pronunciation tests for patients with PD, including the pronunciation of five monosyllabic vowels and a spontaneous dialogue. A triplet multimodel transfer learning network is designed and proposed for identifying subjects with PD in these two groups of tests. First, multisource data extract mel frequency cepstrum coefficient (MFCC) features of speech for preprocessing. Subsequently, a pretrained triplet model represents features from three dimensions as the upstream task of the transfer learning framework. Finally, the pretrained model is reconstructed as a novel model that integrates the triplet model, temporal model, and auxiliary layer as the downstream task, and weights are updated through fine-tuning to identify abnormal speech. Experimental results show that the highest PD detection rates in the two groups of tests are 99% and 90% , respectively, which outperform a large number of internationally popular pattern recognition algorithms and serve as a baseline for other academic researchers in this field.

show abstract

“…Ye et al [11] evaluate multi-scaled gated TCNs with Log Mel spectrograms and emotion causality loss on Ravdess (70.2% unweighted). For augmentation, Pan and Wu [14] use pitch shifting and noise on Ravdess to improve a 77.8% CNN-LSTM baseline, while Abdelhamid et al [15] tune hyperparameters with fractal search on Ravdess Extended.…”

Section: Literature Reviewmentioning

confidence: 99%

A Comparative Analysis of Deep Learning Models for Multi-class Speech Emotion Detection

Elango

2024

Preprint

View full text Add to dashboard Cite

In today's digital age, where communication transcends traditional boundaries, the exploration of deep learning models for Speech Emotion Recognition (SER) holds immense significance. As we increasingly interact through digital platforms, understanding and interpreting emotions becomes crucial. Deep learning models, with their ability to autonomously learn intricate patterns and representations, offer unparalleled potential in enhancing the accuracy and efficiency of SER systems. This project delves into models for multi-class speech emotion recognition on the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). The RAVDESS dataset contains 1440 speech audio recordings from 24 professional actors, expressing 8 different emotions: neutral, calm, happy, sad, angry, fearful, surprise, and disgust. Models including Deep Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory networks (LSTMs), Gated Recurrent Units (GRUs), Temporal Convolutional Networks (TCNs), and ensembles were developed. Additionally, data augmentation through pitch shifting, noise injection, and a combination thereof expanded the dataset. Besides spectrogram inputs, handcrafted audio features like Mel Frequency Cepstral Coefficients (MFCCs), Chroma Short-time Fourier transform, root mean square, and zero crossing rate were experimented with as inputs to further boost model performance. The best-performing models were a Temporal Convolutional Network (TCN), achieving 96.88% testing accuracy, and a Gated Recurrent Unit (GRU) achieving 97.04% testing accuracy in classifying the 8 emotions, outperforming previous benchmark results on this dataset.

show abstract

GM-TCNet: Gated Multi-scale Temporal Convolutional Network using Emotion Causality for Speech Emotion Recognition

Cited by 32 publications

References 57 publications

An Industrial Intrusion Detection Method Based on Hybrid Convolutional Neural Networks with Improved TCN

An Industrial Intrusion Detection Method Based on Hybrid Convolutional Neural Networks with Improved TCN

A Triplet Multimodel Transfer Learning Network for Speech Disorder Screening of Parkinson’s Disease

A Comparative Analysis of Deep Learning Models for Multi-class Speech Emotion Detection

Contact Info

Product

Resources

About