Components loss for neural networks in mask-based speech enhancement

Abstract

Estimating time-frequency domain masks for single-channel speech enhancement using deep learning methods has recently become a popular research field with promising results. In this paper, we propose a novel components loss (CL) for the training of neural networks for mask-based speech enhancement. During the training process, the proposed CL offers separate control over preservation of the speech component quality, suppression of the noise component, and preservation of a naturally sounding residual noise component. We illustrate the potential of the proposed CL by evaluating a standard convolutional neural network (CNN) for mask-based speech enhancement. The new CL is compared to several baseline losses, comprising the conventional mean squared error (MSE) loss w.r.t. speech spectral amplitudes or w.r.t. an ideal-ratio mask, auditory-related loss functions, such as the perceptual evaluation of speech quality (PESQ) loss and the perceptual weighting filter loss, and also the recently proposed SNR loss with two masks. Detailed analysis suggests that the proposed CL obtains a better or at least a more balanced performance across all employed instrumental quality metrics, including SNR improvement, speech component quality, enhanced total speech quality, and particularly also delivers a natural sounding residual noise component. For unseen noise types, we excel even perceptually motivated losses by an about 0.2 points higher PESQ score. The recently proposed so-called SNR loss with two masks not only requires a network with more parameters due to the two decoder heads, but also falls behind on PESQ and POLQA and particularly w.r.t. residual noise quality. Note that the proposed CL shows significantly more 1st ranks among the evaluation metrics than any other baseline. It is easy to implement, and code is provided at https://github.com/ifnspaml/Components-Loss.

Deep Noise Suppression Maximizing Non-Differentiable PESQ Mediated by a Non-Intrusive PESQNet

“…Our previous work denoted as "FCRN/PESQNet, [24]" achieves two 1 st -ranked metrics and one 2 nd rank and significantly outperforms the DNS3 baseline [49] in speech quality measured by PESQ. Under both reverberation conditions, the components loss baseline "FCRN/CL [50]" offers around 0.1 points higher PESQ scores compared to the DNS3 baseline [49], but does not perform so well on DNSMOS. Furthermore, the CL baseline "FCRN/CL [50]" offers the worst dereverberation effects reflected by the lowest SRMR scores among all the baseline methods.…”

“…In [50], a components loss (CL) was proposed for training a mask-based speech enhancement neural network, which offers separate controls over preservation of the speech component quality, suppression of the noise component, and preservation of a natural sounding residual noise component. The experimental results of [50] show improved and balanced performance compared to the conventional MSE loss, the approximated differentiable PESQ loss proposed in [28], and the perceptual weighting filter loss proposed in [30], which is based on code-excited linear predictive (CELP) speech coding. We fine-tune the pre-trained DNS model employing CL on D train DNS3 .…”

“…We fine-tune the pre-trained DNS model employing CL on D train DNS3 . This baseline method is called "FCRN/CL [50]" in the following analysis of the results.…”

“…5) Differentiable PESQ Loss Baseline: In [50], the CL has already shown better performance than the differentiable PESQ loss proposed in [28]. However, we also report on the differentiable PESQ loss [28] as an additional baseline.…”

Time-Domain Joint Training Strategies of Speech Enhancement and Intent Classification Neural Models

“…The network is trained using either ideal binary mask (IBM) or ideal ratio mask (IRM) as training targets [ 20 , 21 ]. Typically, the networks are trained using the mean squared error (MSE) either on the masks or on the reconstructed signal [ 22 , 23 ]. Despite the promising performance achievable in terms of SDR and intelligibility, the presence of artifacts in the reconstructed signals compromises the performance of further processing stages.…”