Spatio-Temporal Features Representation Using Recurrent Capsules for Monaural Speech Enhancement

Abstract

Single-channel speech enhancement is important for modern communication systems and has received a lot of attention. A convolutional neural network (CNN) successfully learns feature representations from speech spectrograms but loses spatial information due to distortion, which is important for humans to understand speech. Speech feature learning is an important ongoing research to capture higher-level representations of speech that go beyond conventional techniques. By considering the hierarchical structure and temporal relationships within speech signals, capsule networks (CapsNets) have the potential to provide more expressive and context-aware feature representations. By considering the advantages of CapNets over CNN, this study presents a model for monaural speech enhancement that keeps spatial information in a capsule and uses dynamic routing to pass it to higher layers. Dynamic routing replaces the pooling recurrent hidden states to get speech features from the outputs of the capsule. Leveraging long-term contexts provides identification of the target speaker. Therefore, a gated recurrent layer, gated recurrent unit (GRU), or long-short-term memory (LSTM), is placed above the CNN module and next to the capsule module in the architecture. This makes it viable to extract spatial features and long-term temporal dynamics. The suggested convolutional recurrent CapNet performs better compared to the models based on CNNs and recurrent neural networks. The suggested speech enhancement produces considerably better speech quality and intelligibility. With the LibriSpeech and VoiceBank+DEMAND databases, the suggested speech enhancement improves the intelligibility and quality by 18.33% and (0.94) 36.82% over the noisy mixtures.

A convolutional neural network with image and numerical data to improve farming of edible crickets as a source of food—A decision support system

“…The x-axis represents time of sequences of spectra, and color brightness on the other axis represents the frequency of the strength of each component at each time frame. Spectrograms show where there is high or low energy, and how energy levels vary over time (Ali et al, 2024 ). In insect song synthesis, spectrograms capture the temporal and spectral characteristics of the insect sounds.…”

Deep Learning: Historical Overview from Inception to Actualization, Models, Applications and Future Trends

“…When combined with recurrent features of Recurrent Neural Networks (RNN), this hybridization can be useful in environments that require memory over time. Studies such as (Wu et al, 2021;Sezavar et al, 2024;Ali et al, 2024) have demonstrated the optimal performance of Cap-sNets with recurrent structures.…”