INTERSPEECH 2020 Deep Noise Suppression Challenge: A Fully Convolutional Recurrent Network (FCRN) for Joint Dereverberation and Denoising

Strake, Maximilian; Defraene, Bruno; Fluyt, Kristoff; Tirry, Wouter; Fingscheidt, Tim

doi:10.21437/interspeech.2020-2439

Cited by 16 publications

(39 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Parts of this work, namely one of our two proposed losses, have been pre-published with limited analysis and evaluation in [49]. Following up our proposed CLs, Xia et al proposed a modified components loss for speech enhancement in [50], and Strake et al proposed a component loss for a joint denoising and dereverberation task in [51].…”

Section: Introductionmentioning

confidence: 99%

Components loss for neural networks in mask-based speech enhancement

Elshamy

Zhao

et al. 2021

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

Estimating time-frequency domain masks for single-channel speech enhancement using deep learning methods has recently become a popular research field with promising results. In this paper, we propose a novel components loss (CL) for the training of neural networks for mask-based speech enhancement. During the training process, the proposed CL offers separate control over preservation of the speech component quality, suppression of the noise component, and preservation of a naturally sounding residual noise component. We illustrate the potential of the proposed CL by evaluating a standard convolutional neural network (CNN) for mask-based speech enhancement. The new CL is compared to several baseline losses, comprising the conventional mean squared error (MSE) loss w.r.t. speech spectral amplitudes or w.r.t. an ideal-ratio mask, auditory-related loss functions, such as the perceptual evaluation of speech quality (PESQ) loss and the perceptual weighting filter loss, and also the recently proposed SNR loss with two masks. Detailed analysis suggests that the proposed CL obtains a better or at least a more balanced performance across all employed instrumental quality metrics, including SNR improvement, speech component quality, enhanced total speech quality, and particularly also delivers a natural sounding residual noise component. For unseen noise types, we excel even perceptually motivated losses by an about 0.2 points higher PESQ score. The recently proposed so-called SNR loss with two masks not only requires a network with more parameters due to the two decoder heads, but also falls behind on PESQ and POLQA and particularly w.r.t. residual noise quality. Note that the proposed CL shows significantly more 1st ranks among the evaluation metrics than any other baseline. It is easy to implement, and code is provided at https://github.com/ifnspaml/Components-Loss.

show abstract

Section: Introductionmentioning

confidence: 99%

Components loss for neural networks in mask-based speech enhancement

Elshamy

Zhao

et al. 2021

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Due to the promising performance of convolutional encoderdecoder topology in the SE applications [20,21,22], we employ it in the first three modules. As both DN-Net and DR-Net are operated in the magnitude domain, only one decoder is utilized to recover the magnitude.…”

Section: Network Configurationsmentioning

confidence: 99%

A Simultaneous Denoising and Dereverberation Framework with Target Decoupling

Li¹,

Liu²,

Luo³

et al. 2021

Preprint

View full text Add to dashboard Cite

Background noise and room reverberation are regarded as two major factors to degrade the subjective speech quality. In this paper, we propose an integrated framework to address simultaneous denoising and dereverberation under complicated scenario environments. It adopts a chain optimization strategy and designs four sub-stages accordingly. In the first two stages, we decouple the multi-task learning w.r.t. complex spectrum into magnitude and phase, and only implement noise and reverberation removal in the magnitude domain. Based on the estimated priors above, we further polish the spectrum in the third stage, where both magnitude and phase information are explicitly repaired with the residual learning. Due to the data mismatch and nonlinear effect of DNNs, the residual noise often exists in the DNN-processed spectrum. To resolve the problem, we adopt a light-weight algorithm as the post-processing module to capture and suppress the residual noise in the non-active regions. In the Interspeech 2021 Deep Noise Suppression (DNS) Challenge, our submitted system ranked top-1 for the real-time track in terms of Mean Opinion Score (MOS) with ITU-T P.835 framework.

show abstract

“…The Deep Noise Suppression (DNS) Challenge for Interspeech 2020 (DNS1) [18], ICASSP 2021 (DNS2) [19], and Interspeech 2021 (DNS3) [20] addressed a challenging task for joint dereverberation and denoising. Strake et al [21] achieved a second rank in the DNS1 non-realtime track by training an actually realtime-capable FCRN for complex spectral maskingbased denoising and dereverberation, which we build upon. All of the abovementioned works apply supervised training using synthetic data, where clean speech and noise are prepared separately and are mixed by addition, while reverberation (if considered) is applied by convolution with simulated or recorded room impulse responses (RIRs).…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, we solved the problems addressed in [29] by training the FCRN and the PESQNet alternatingly, similar to alternating training schemes used in adversarial trainings [22][23][24]. We combine the loss provided by the PESQNet and the successful FCRN loss from [21], thereby training a complex mask-based FCRN in a "weakly" supervised way, employing both synthetic and real data in training.…”

Section: Introductionmentioning

confidence: 99%

Deep Noise Suppression With Non-Intrusive PESQNet Supervision Enabling the Use of Real Training Data

Xu¹,

Strake²,

Fingscheidt³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Data-driven speech enhancement employing deep neural networks (DNNs) can provide state-of-the-art performance even in the presence of non-stationary noise. During the training process, most of the speech enhancement neural networks are trained in a fully supervised way with losses requiring noisy speech to be synthesized by clean speech and additive noise. However, in a real implementation, only the noisy speech mixture is available, which leads to the question, how such data could be advantageously employed in training. In this work, we propose an end-to-end non-intrusive PESQNet DNN which estimates perceptual evaluation of speech quality (PESQ) scores, allowing a reference-free loss for real data. As a further novelty, we combine the PESQNet loss with denoising and dereverberation loss terms, and train a complex mask-based fully convolutional recurrent neural network (FCRN) in a "weakly" supervised way, each training cycle employing some synthetic data, some real data, and again synthetic data to keep the PESQNet up-to-date. In a subjective listening test, our proposed framework outperforms the Interspeech 2021 Deep Noise Suppression (DNS) Challenge baseline overall by 0.09 MOS points and in particular by 0.45 background noise MOS points.

show abstract

INTERSPEECH 2020 Deep Noise Suppression Challenge: A Fully Convolutional Recurrent Network (FCRN) for Joint Dereverberation and Denoising

Cited by 16 publications

References 22 publications

Components loss for neural networks in mask-based speech enhancement

Components loss for neural networks in mask-based speech enhancement

A Simultaneous Denoising and Dereverberation Framework with Target Decoupling

Deep Noise Suppression With Non-Intrusive PESQNet Supervision Enabling the Use of Real Training Data

Contact Info

Product

Resources

About