On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

Abstract

Many deep learning-based speech enhancement algorithms are designed to minimize the mean-square error (MSE) in some transform domain between a predicted and a target speech signal. However, optimizing for MSE does not necessarily guarantee high speech quality or intelligibility, which is the ultimate goal of many speech enhancement algorithms. Additionally, only little is known about the impact of the loss function on the emerging class of time-domain deep learning-based speech enhancement systems.We study how popular loss functions influence the performance of deep learning-based speech enhancement systems. First, we demonstrate that perceptually inspired loss functions might be advantageous if the receiver is the human auditory system. Furthermore, we show that the learning rate is a crucial design parameter even for adaptive gradient-based optimizers, which has been generally overlooked in the literature. Also, we found that waveform matching performance metrics must be used with caution as they in certain situations can fail completely. Finally, we show that a loss function based on scale-invariant signalto-distortion ratio (SI-SDR) achieves good general performance across a range of popular speech enhancement evaluation metrics, which suggests that SI-SDR is a good candidate as a generalpurpose loss function for speech enhancement systems.

A Supervised Speech Enhancement Approach with Residual Noise Control for Voice Communication

“…U-Net is chosen as the network in this letter, which has been widely adopted for speech separation task [30]. As shown in Fig.…”

“…This letter chooses three loss functions including MSE in (4), Time-MSE-based loss (TMSE) [30] and recently proposed SI-SDR-based loss [30] as baselines. AS T-F domainbased network is used, an additional fixed iSTFT-like layer is needed to transform the estimated T-F spectrum back into time domain for TMSE-and SI-SDR-based loss [31].…”

WaveCRN: An Efficient Convolutional Recurrent Neural Network for End-to-End Speech Enhancement

“…Another class of SE methods proposes to directly perform enhancement on the raw waveform [27]- [31], which are generally called waveform-mapping-based approaches. Among the deep learning models, fully convolutional networks (FCNs) have been widely used to directly perform waveform mapping [28], [32]- [34].…”

End-To-End Non-Negative Autoencoders for Sound Source Separation

“…With the current rapid strides in neural networks (NNs) and deep learning, several sophisticated architectures have been proposed and successfully used for single-channel source separation [1,2,3,4]. More recently, we have started to operate directly on the waveforms with several end-to-end approaches available [2,5,6], and use better cost-functions motivated by the Source-to-Distortion ratio (SDR) [7,8,9,10,11,2]. Using deep-clustering [1] and permutation-invariant training [12], we can train the networks to perform speaker-independent source separation.…”