DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement

Abstract

Speech enhancement has benefited from the success of deep learning in terms of intelligibility and perceptual quality. Conventional time-frequency (TF) domain methods focus on predicting TF-masks or speech spectrum, via a naive convolution neural network (CNN) or recurrent neural network (RNN). Some recent studies use complex-valued spectrogram as a training target but train in a real-valued network, predicting the magnitude and phase component or real and imaginary part, respectively. Particularly, convolution recurrent network (CRN) integrates a convolutional encoder-decoder (CED) structure and long short-term memory (LSTM), which has been proven to be helpful for complex targets. In order to train the complex target more effectively, in this paper, we design a new network structure simulating the complex-valued operation, called Deep Complex Convolution Recurrent Network (DCCRN), where both CNN and RNN structures can handle complex-valued operation. The proposed DCCRN models are very competitive over other previous networks, either on objective or subjective metric. With only 3.7M parameters, our DCCRN models submitted to the Interspeech 2020 Deep Noise Suppression (DNS) challenge ranked first for the real-time-track and second for the non-real-time track in terms of Mean Opinion Score (MOS).

HGCN: Harmonic gated compensation network for speech enhancement

“…The decoder predicts a complex ratio mask M = Cat(M r , M i ) ∈ R T ×2F , where M r and M i represent the real and imaginary parts of mask. We use the mask applying scheme of DCCRN-E [3], which is called Mask Apply E in Fig. 1,…”

“…T models process the waveform directly to obtain the target speech [1]. T-F models precess the spectrum after the short-time fast Fourier transform (STFT) [2][3][4]. Generally speaking, for speech enhancement, it's the T-F structure of speech that is enhanced.…”

Hybrid Neural Networks for On-device Directional Hearing

“…For example, complex multiplication capture rotation in the complex domain and can easily manipulate the signal phase. Thus, complex neural networks are found to be more effective for applications such as wireless communication (Marseet and Sahin 2017) and noise suppression (Hu et al 2020;Bassey, Qian, and Li 2021). Compared to realvalued networks, complex representation also restricts the degree of freedom of the parameters by enforcing correlation between the real and imaginary parts, which enhances the generalization capacity of the model in other applications.…”

Mixed Precision DNN Qunatization for Overlapped Speech Separation and Recognition

“…Previous researches suggest that the complex ratio masks (CRMs) outperform both the binary masks (BMs) and real-value ratio masks (RMs) on speech separation [30,31] and enhancement [32] tasks. For this reason, the complex ideal ratio mask (cIRM) m t,f of the target speech is estimated in the separation module.…”