Determining speaker attributes from stress-affected speech in emergency situations with hybrid SVM-DNN architecture

Ahmad, Jamil; Sajjad, Muhammad; Rho, Seungmin; Kwon, Soonil; Lee, Mi Young; Baik, Sung Wook

doi:10.1007/s11042-016-4041-7

Cited by 10 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Generally speaking, a SER system is composed of two parts: a preprocessing part that extracts suitable features and a classifier that employs those features to perform ER. This section overviews existing strategies in the SER research area [ 21 , 22 ].…”

Section: Methodsmentioning

confidence: 99%

3D CNN-Based Speech Emotion Recognition Using K-Means Clustering and Spectrograms

Hajarolasvadi

Demirel

2019

Entropy

130

View full text Add to dashboard Cite

Detecting human intentions and emotions helps improve human–robot interactions. Emotion recognition has been a challenging research direction in the past decade. This paper proposes an emotion recognition system based on analysis of speech signals. Firstly, we split each speech signal into overlapping frames of the same length. Next, we extract an 88-dimensional vector of audio features including Mel Frequency Cepstral Coefficients (MFCC), pitch, and intensity for each of the respective frames. In parallel, the spectrogram of each frame is generated. In the final preprocessing step, by applying k-means clustering on the extracted features of all frames of each audio signal, we select k most discriminant frames, namely keyframes, to summarize the speech signal. Then, the sequence of the corresponding spectrograms of keyframes is encapsulated in a 3D tensor. These tensors are used to train and test a 3D Convolutional Neural network using a 10-fold cross-validation approach. The proposed 3D CNN has two convolutional layers and one fully connected layer. Experiments are conducted on the Surrey Audio-Visual Expressed Emotion (SAVEE), Ryerson Multimedia Laboratory (RML), and eNTERFACE’05 databases. The results are superior to the state-of-the-art methods reported in the literature.

show abstract

Section: Methodsmentioning

confidence: 99%

3D CNN-Based Speech Emotion Recognition Using K-Means Clustering and Spectrograms

Hajarolasvadi

Demirel

2019

Entropy

130

View full text Add to dashboard Cite

show abstract

“…With the combination of these features, which the authors achieved an 85.8% accuracy with the testing set of the EMO-DB [27] corpus. Moreover, the researchers used the SVM classifier with different type features [41] that included the extracted timber and the MFCCs features [14] and the extracted Fourier parameter and the MFCCs features, and they classified them by the SVM classifier that achieved 83.93% and 73.3% accuracy respectively, on the EMO-DB [27] dataset.…”

Section: Related Workmentioning

confidence: 99%

CLSTM: Deep Feature-Based Speech Emotion Recognition Using the Hierarchical ConvLSTM Network

2020

View full text Add to dashboard Cite

Artificial intelligence, deep learning, and machine learning are dominant sources to use in order to make a system smarter. Nowadays, the smart speech emotion recognition (SER) system is a basic necessity and an emerging research area of digital audio signal processing. However, SER plays an important role with many applications that are related to human–computer interactions (HCI). The existing state-of-the-art SER system has a quite low prediction performance, which needs improvement in order to make it feasible for the real-time commercial applications. The key reason for the low accuracy and the poor prediction rate is the scarceness of the data and a model configuration, which is the most challenging task to build a robust machine learning technique. In this paper, we addressed the limitations of the existing SER systems and proposed a unique artificial intelligence (AI) based system structure for the SER that utilizes the hierarchical blocks of the convolutional long short-term memory (ConvLSTM) with sequence learning. We designed four blocks of ConvLSTM, which is called the local features learning block (LFLB), in order to extract the local emotional features in a hierarchical correlation. The ConvLSTM layers are adopted for input-to-state and state-to-state transition in order to extract the spatial cues by utilizing the convolution operations. We placed four LFLBs in order to extract the spatiotemporal cues in the hierarchical correlational form speech signals using the residual learning strategy. Furthermore, we utilized a novel sequence learning strategy in order to extract the global information and adaptively adjust the relevant global feature weights according to the correlation of the input features. Finally, we used the center loss function with the softmax loss in order to produce the probability of the classes. The center loss increases the final classification results and ensures an accurate prediction as well as shows a conspicuous role in the whole proposed SER scheme. We tested the proposed system over two standard, interactive emotional dyadic motion capture (IEMOCAP) and ryerson audio visual database of emotional speech and song (RAVDESS) speech corpora, and obtained a 75% and an 80% recognition rate, respectively.

show abstract

“…Sistem pengenalan emosi wicara dapat digunakan oleh penyandang disablitas untuk komunikasi [3], [4], oleh aktor untuk konsistensi wicara emosi serta untuk acara pada media televisi yang interaktif [5], untuk membangun model guru dalam bentuk virtual [6], juga digunakan dalam studi yang mendeteksi kerusakan otak pada manusia [7], dan desain canggih dari speech embedding [8], [9]. Salah satu contoh mengenai emosi lainnya adalah bahwa emosi mengatur kehidupan kita sehari-hari; emosi merupakan bagian besar dari pengalaman manusia dan mempengaruhi pengambilan keputusan kita [10]- [12]. Untuk mencapai tujuan ambisius seperti itu, pengumpulan database wicara emosional adalah prasyarat.…”

Section: Pendahuluanunclassified

Deteksi Emosi Wicara pada Media On-Demand menggunakan SVM dan LSTM

Ainurrochman

Adi

Gumelar

2020

RESTI

View full text Add to dashboard Cite

To date, there are many speech data sets with emotional classes, but with impromptu or intentional actors. The native speakers are given a stimulus in each emotion expression. Because natural conversation from secretly recorded daily communication still raises ethical issues, then using voice data that takes samples from movies and podcasts is the most appropriate step to take the best insights from speech. Professional actors are trained to induce the most real emotions close to natural, through the Stanislavski acting method. The speech dataset that meets this qualification is the Human voice Natural Language from On-demand media (HENLO). Within HENLO, there are basic per-emotion audio clips of films and podcasts originating from Media On-Demand, a motion video entertainment media platform with the freedom to play and download at any time. In this paper, we describe the use of sound clips from HENLO, then conduct learning using Support Vector Machine (SVM) and Long Short-Term Memory (LSTM). In these two methods, we found the best strategy by training LSTMs first, then then feeding the model to SVM, with a data split strategy at 80:20 scale. The results of the five training phases show that the last accuracy results increased by more than 17% compared to the first training. These results mean both complement and methods are important for improving classification accuracy.

show abstract

Determining speaker attributes from stress-affected speech in emergency situations with hybrid SVM-DNN architecture

Cited by 10 publications

References 22 publications

3D CNN-Based Speech Emotion Recognition Using K-Means Clustering and Spectrograms

3D CNN-Based Speech Emotion Recognition Using K-Means Clustering and Spectrograms

CLSTM: Deep Feature-Based Speech Emotion Recognition Using the Hierarchical ConvLSTM Network

Deteksi Emosi Wicara pada Media On-Demand menggunakan SVM dan LSTM

Contact Info

Product

Resources

About