Fusion of mel and gammatone frequency cepstral coefficients for speech emotion recognition using deep C-RNN

Kumaran, U; Rammohan, S. Radha; Nagarajan, Senthil Murugan; Prathik, A.

doi:10.1007/s10772-020-09792-x

Cited by 64 publications

(10 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This idea has profoundly impacted subsequent research on effective teaching and learning. Kumaran et al later proposed a classification theory of educational goals, which requires a classification system based on emotions [ 10 ]. The classification dimensions are mainly reflected in five levels: acceptance, response, value judgment, organization, and characterization of value and value complexes, all of which have specific emotional meanings and sublevels corresponding to the level.…”

Section: Related Workmentioning

confidence: 99%

A Classroom Emotion Recognition Model Based on a Convolutional Neural Network Speech Emotion Algorithm

Yuan

2022

Occupational Therapy International

View full text Add to dashboard Cite

In this paper, we construct a model of convolutional neural network speech emotion algorithm, analyze the classroom identified by the neural network with a certain degree of confidence together with the school used in the dataset, find the characteristics and rules of teachers’ control of classroom emotion nowadays using big data, find the parts of classroom emotion, and design a classroom emotion recognition model based on convolutional neural network speech emotion algorithm according to these characteristics. This paper will investigate the factors and patterns of teachers’ emotional control in the classroom. In this paper, the existing neural network is adapted and improved, and some preprocessing is performed on the current dataset to train the network. The network used in this paper is a combination of convolutional neural network (CNN) and recurrent neural network (RNN), which takes advantage of both CNN for feature extraction and RNN for memory capability in the sequence model. This network has a good effect on both object labeling and speech recognition. For the problem of extracting emotion features of whole-sentence speech, we propose an attention mechanism-based emotion recognition algorithm for variable-length speech and design a spatiotemporal attention module for the speech emotion algorithm and a convolutional channel attention module for the CNN network to reduce the contribution of the spatiotemporal data of the speech emotion algorithm and the unimportant parts of the CNN convolutional channel feature data in the subsequent recognition by the attention mechanism. In turn, the weight of core key data and features is increased to improve the model recognition accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

A Classroom Emotion Recognition Model Based on a Convolutional Neural Network Speech Emotion Algorithm

Yuan

2022

Occupational Therapy International

View full text Add to dashboard Cite

show abstract

“…The literature [17] optimized the inception structure by incorporating LSTM and proposed OI-LSTM model, which has excellent recognition effect and also the model has good fault tolerance. The literature [18] proposed the recursive multilevel fusion network (RMFN), which decomposes the spatio-temporal fusion problem into multiple stages, each focusing on a subset of multimodal signals for specialized and efficient fusion. The literature [19] proposes a multimodal information fuzzy fusion algorithm.…”

Section: Related Workmentioning

confidence: 99%

English Pronunciation Standards Based on Multimodal Acoustic Sensors

Zhu

2021

Journal of Sensors

View full text Add to dashboard Cite

In recent years, economic globalization is the trend, and communication between countries is getting closer and closer; more and more people begin to pay attention to learning spoken English. The development of computer-aided language learning makes it more convenient for people to learn spoken English; however, the detection and correction of incorrect English pronunciation, which is its core, are still inadequate. In this paper, we propose a multimodal end-to-end English pronunciation error detection and correction model based on audio and video, which does not require phoneme forced alignment of the English pronunciation video signal to be processed, and uses rich audio and video features for English pronunciation error detection, which improves the error detection accuracy to a great extent especially in noisy environments. To address the shortcomings of the current lip feature extraction algorithm which is too complicated and not enough characterization ability, a feature extraction scheme based on the lip opening and closing angle is proposed. The lip syllable frames are obtained by video frame splitting, the syllables are denoised, the key point information of the lips is obtained using a gradient enhancement-based regression tree algorithm, the effects of speaker tilt and movement are removed by scale normalization, and finally, the lip opening and closing angles are calculated using mathematical geometry, and the lip feature values are generated by combining the angle changes.

show abstract

“…To make Sphinx-4 be able to recognize concatenation, it first needs to be able to accept concatenated grammar and then build a search grid based on the input grammar nodes, so that when the input speech contains concatenated components, it can be detected and recognized by the system. This is achieved by adding methods to the original class that can dynamically generate new grammar nodes so that the system can accept grammar nodes processed by concatenation rules [25]. The more important parameter in the process of fundamental frequency feature extraction is the setting of the frequency range of the band-pass filter, which is set to 52 and 620, respectively, and the window function adopts Gaussian function.…”

Section: Experimental Design For Automatic Assessment Of Spokenmentioning

confidence: 99%

An Automatic Assessment Method for Spoken English Based on Multimodal Feature Fusion

Zhang

2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

This paper provides a comprehensive explanation of the theoretical foundations of multimodal discourse analysis theory as applied to speaking instructional design. The specific application of multimodal theory in the teaching of elementary English speaking classrooms is explored through the teaching design of elementary English speaking classrooms, the teaching implementation of multimodal teaching design is carried out, and the effect of the teaching practice of elementary English speaking guided by multimodal discourse analysis theory is comprehensively evaluated through classroom observation method, questionnaire survey method, and interview method, combined with the teaching evaluation and teaching implementation effect, which is the multimodal teaching design. The paper also summarizes the findings and shortcomings of the study. Through the teaching design and implementation, the advantages of multimodal teaching are obvious; it can combine with modern advanced teaching techniques to create more realistic communicative situations in the classroom, gather and present various modal resources and information, and ensure rich and diverse language input; students can receive various sensory stimuli in the classroom, deepen their memory and experience of language, increase the interest of classroom teaching, and improve students’ participation. It also increases the interest of the classroom and enhances students’ participation and motivation. Based on multimodal theory, the author designed a multimodal teaching framework for a semester-long speaking course in the speaking classroom for reference. The fuzzy measures were constructed based on subsets of language segments containing 10 phonemes belonging to the same HDP set. Finally, linguistic scores are given by the Surgeon integral model based on the plausibility of the system and the fuzzy measures. The experimental results based on Sphinx-4 show that the evaluation model yields plausible and stable evaluation results for the 3 test sets at an average correct recognition rate of 84.7% of phonemes.

show abstract

Fusion of mel and gammatone frequency cepstral coefficients for speech emotion recognition using deep C-RNN

Cited by 64 publications

References 29 publications

A Classroom Emotion Recognition Model Based on a Convolutional Neural Network Speech Emotion Algorithm

A Classroom Emotion Recognition Model Based on a Convolutional Neural Network Speech Emotion Algorithm

English Pronunciation Standards Based on Multimodal Acoustic Sensors

An Automatic Assessment Method for Spoken English Based on Multimodal Feature Fusion

Contact Info

Product

Resources

About