The Application of Capsule Neural Network Based CNN for Speech Emotion Recognition

Xin-Cheng, Wen,; Liu, Kun-Hong; Zhang, Weiming; Jiang, Kezhi

doi:10.1109/icpr48806.2021.9412360

Cited by 10 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section presents an overview of the existing research in the field of speech emotion recognition. Wen et al (2020) [1] presented a fusion model that combined the strengths of a capsule network and a Convolutional Neural Network, denoted as CapCNN. The method involved a two-step pre-processing procedure comprising voice ac-tivity detection and a windowed framework.…”

Section: Related Workmentioning

confidence: 99%

MaxMViT-MLP: Multiaxis and Multiscale Vision Transformers Fusion Network for Speech Emotion Recognition

Ong,

Lee,

Lim

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Vision Transformers, known for their innovative architectural design and modeling capabilities, have gained significant attention in computer vision. This paper presents a dual-path approach that leverages the strengths of the Multi-Axis Vision Transformer (MaxViT) and the Improved Multiscale Vision Transformer (MViTv2). It starts by encoding speech signals into Constant-Q Transform (CQT) spectrograms and Mel Spectrograms with Short-Time Fourier Transform (Mel-STFT). The CQT spectrogram is then fed into the MaxViT model, while the Mel-STFT is input to the MViTv2 model to extract informative features from the spectrograms. These features are integrated and passed into a Multilayer Perceptron (MLP) model for final classification. This hybrid model is named the "MaxViT and MViTv2 Fusion Network with Multilayer Perceptron (MaxMViT-MLP)." The MaxMViT-MLP model achieves remarkable results with an accuracy of 95.28% on the Emo-DB, 89.12% on the RAVDESS dataset, and 68.39% on the IEMOCAP dataset, substantiating the advantages of integrating multiple audio feature representations and Vision Transformers in speech emotion recognition.

show abstract

Section: Related Workmentioning

confidence: 99%

MaxMViT-MLP: Multiaxis and Multiscale Vision Transformers Fusion Network for Speech Emotion Recognition

Ong,

Lee,

Lim

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Convolutional neural networks (CNNs) are the primary tool for solving various computer vision problems: pattern recognition [1], detection of different objects [2], [3], semantic segmentation [4] and many others. Although new transformerbased [5], [6] or deep MLP-based [7], [8] neural networks sometimes outperform CNNs on challenging datasets, they are usually harder to train, have more parameters and require more computational resources for inference [8], [9].…”

Section: Introductionmentioning

confidence: 99%

Fast matrix multiplication for binary and ternary CNNs on ARM CPU

Trusov¹,

Limonova²,

Nikolaev³

et al. 2022

Preprint

View full text Add to dashboard Cite

Low-bit quantized neural networks (QNNs) are of great interest in practical applications because they significantly reduce the consumption of both memory and computational resources. Binary neural networks (BNNs) are memory and computationally efficient as they require only one bit per weight and activation and can be computed using Boolean logic and bit count operations. QNNs with ternary weights and activations (TNNs) and binary weights and ternary activations (TBNs) aim to improve recognition quality compared to BNNs while preserving low bit-width. However, their efficient implementation is usually considered on ASICs and FPGAs, limiting their applicability in real-life tasks. At the same time, one of the areas where efficient recognition is most in demand is recognition on mobile devices using their CPUs. However, there are no known fast implementations of TBNs and TNN, only the daBNN library for BNNs inference. In this paper, we propose novel fast algorithms of ternary, ternary-binary, and binary matrix multiplication for mobile devices with ARM architecture. In our algorithms, ternary weights are represented using 2-bit encoding and binary -using one bit. It allows us to replace matrix multiplication with Boolean logic operations that can be computed on 128bits simultaneously, using ARM NEON SIMD extension. The matrix multiplication results are accumulated in 16-bit integer registers. We also use special reordering of values in left and right matrices. All that allows us to efficiently compute a matrix product while minimizing the number of loads and stores compared to the algorithm from daBNN. Our algorithms can be used to implement inference of convolutional and fully connected layers of TNNs, TBNs, and BNNs. We evaluate them experimentally on ARM Cortex-A73 CPU and compare their inference speed to efficient implementations of full-precision, 8bit, and 4-bit quantized matrix multiplications. Our experiment shows our implementations of ternary and ternary-binary matrix multiplications to have almost the same inference time, and they are 3.6 times faster than full-precision, 2.5 times faster than 8bit quantized, and 1.4 times faster than 4-bit quantized matrix multiplication but 2.9 slower than binary matrix multiplication.

show abstract

An Improved Capsule Network for Speech Emotion Recognition

Zhang

Huang

2022

Communications in Computer and Information Science

View full text Add to dashboard Cite

The Application of Capsule Neural Network Based CNN for Speech Emotion Recognition

Cited by 10 publications

References 17 publications

MaxMViT-MLP: Multiaxis and Multiscale Vision Transformers Fusion Network for Speech Emotion Recognition

MaxMViT-MLP: Multiaxis and Multiscale Vision Transformers Fusion Network for Speech Emotion Recognition

Fast matrix multiplication for binary and ternary CNNs on ARM CPU

An Improved Capsule Network for Speech Emotion Recognition

Contact Info

Product

Resources

About