Speech Intelligibility Potential of General and Specialized Deep Neural Network Based Speech Enhancement Systems

Kolbk, Morten; Tan, Zheng‐Hua; Jensen, Jesper

doi:10.1109/taslp.2016.2628641

Cited by 187 publications

(114 citation statements)

References 53 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…The third loss function we consider is based on the short-time objective intelligibility (STOI) speech intelligibility estimator [32]. STOI is currently the, perhaps, most commonly used speech intelligibility estimator for objectively evaluating the performance of speech enhancement systems [6], [7], [9], [13]. This is presumably driven by the fact that STOI has proven to be able to quite accurately predict the intelligibility of noisy/processed speech in a large range of acoustic scenarios, including ideal time-frequency weighted noisy speech [32] and noisy speech enhanced by single-microphone time-frequency weighting-based speech enhancement systems [32] (se also [33], [56]).…”

Section: Short-time Objective Intelligibilitymentioning

confidence: 99%

See 1 more Smart Citation

On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

Kolbæk

Tan

Jensen

et al. 2020

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

115

View full text Add to dashboard Cite

Many deep learning-based speech enhancement algorithms are designed to minimize the mean-square error (MSE) in some transform domain between a predicted and a target speech signal. However, optimizing for MSE does not necessarily guarantee high speech quality or intelligibility, which is the ultimate goal of many speech enhancement algorithms. Additionally, only little is known about the impact of the loss function on the emerging class of time-domain deep learning-based speech enhancement systems.We study how popular loss functions influence the performance of deep learning-based speech enhancement systems. First, we demonstrate that perceptually inspired loss functions might be advantageous if the receiver is the human auditory system. Furthermore, we show that the learning rate is a crucial design parameter even for adaptive gradient-based optimizers, which has been generally overlooked in the literature. Also, we found that waveform matching performance metrics must be used with caution as they in certain situations can fail completely. Finally, we show that a loss function based on scale-invariant signalto-distortion ratio (SI-SDR) achieves good general performance across a range of popular speech enhancement evaluation metrics, which suggests that SI-SDR is a good candidate as a generalpurpose loss function for speech enhancement systems.

show abstract

Section: Short-time Objective Intelligibilitymentioning

confidence: 99%

“…This subset of WSJ0 consists in total of 11613 utterances approximately equally divided among 44 male speakers and 47 female speakers. This ensures that the training dataset contains a large speaker variability, which allows the final speech enhancement system to be largely speaker independent [9].…”

Section: A Noise-free Speech Mixturesmentioning

confidence: 99%

On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

Kolbæk

Tan

Jensen

et al. 2020

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

115

View full text Add to dashboard Cite

show abstract

“…In [54] it is found that the AFE enhancement method outperforms MMSE-based methods for noise-robust speech recognition. Recently DNN based speech enhancement methods have also been proposed for improving speech intelligibility [55], automatic speech recognition [56] and speaker verification [57,58].…”

Section: Robust Vad In Noisementioning

confidence: 99%

rVAD: An unsupervised segment-based robust voice activity detection method

Tan

Sarkar

Dehak

2020

Computer Speech & Language

Self Cite

109

View full text Add to dashboard Cite

This paper presents an unsupervised segment-based method for robust voice activity detection (rVAD). The method consists of two passes of denoising followed by a voice activity detection (VAD) stage. In the first pass, high-energy segments in a speech signal are detected by using a posteriori signal-to-noise ratio (SNR) weighted energy difference and if no pitch is detected within a segment, the segment is considered as a high-energy noise segment and set to zero. In the second pass, the speech signal is denoised by a speech enhancement method, for which several methods are explored. Next, neighbouring frames with pitch are grouped together to form pitch segments, and based on speech statistics, the pitch segments are further extended from both ends in order to include both voiced and unvoiced sounds and likely non-speech parts as well. In the end, a posteriori SNR weighted energy difference is applied to the extended pitch segments of the denoised speech signal for detecting voice activity. We evaluate the VAD performance of the proposed method using two databases, RATS and Aurora-2, which contain a large variety of noise conditions. The rVAD method is further evaluated, in terms of speaker verification performance, on the RedDots 2016 challenge database and its noise-corrupted versions. Experiment results show that rVAD is compared favourably with a number of existing methods. In addition, we present a modified version of rVAD where computationally intensive pitch extraction is replaced by computationally efficient spectral flatness calculation. The modified version significantly reduces the computational complexity at the cost of moderately inferior VAD performance, which is an advantage when processing a large amount of data and running on low resource devices. The source code of rVAD is made publicly available.

show abstract

“…However, a significant amount of research has focused on the offline setting with many algorithms being unsuitable for real-time use due to batch processing or computational requirements. Recent speech enhancement and source separation approaches based on deep neural networks offer impressive performance gains compared with traditional real-time signal processing methods [1]- [4], however these methods tend to be computationally demanding, preventing their use in low-power devices, and often rely on future information, preventing their use in realtime systems. Deep learning methods also require a significant S. Wood and J. Rouat are affiliated with NECOTIS, Department of Electrical and Computer Engineering, University of Sherbrooke, Sherbrooke, QC, J1K 2R1, Canada.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Low Latency Speech Enhancement With RT-GCC-NMF

Wood

Rouat

2019

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

In this paper, we present RT-GCC-NMF: a realtime (RT), two-channel blind speech enhancement algorithm that combines the non-negative matrix factorization (NMF) dictionary learning algorithm with the generalized cross-correlation (GCC) spatial localization method. Using a pre-learned universal NMF dictionary, RT-GCC-NMF operates in a frame-by-frame fashion by associating individual dictionary atoms to target speech or background interference based on their estimated time-delay of arrivals (TDOA). We evaluate RT-GCC-NMF on two-channel mixtures of speech and real-world noise from the Signal Separation and Evaluation Campaign (SiSEC). We demonstrate that this approach generalizes to new speakers, acoustic environments, and recording setups from very little training data, and outperforms all but one of the algorithms from the SiSEC challenge in terms of overall Perceptual Evaluation methods for Audio Source Separation (PEASS) scores and compares favourably to the ideal binary mask baseline. Over a wide range of input SNRs, we show that this approach simultaneously improves the PEASS and signal to noise ratio (SNR)-based Blind Source Separation (BSS) Eval objective quality metrics as well as the short-time objective intelligibility (STOI) and extended STOI (ESTOI) objective speech intelligibility metrics. A flexible, soft masking function in the space of NMF activation coefficients offers real-time control of the trade-off between interference suppression and target speaker fidelity. Finally, we use an asymmetric short-time Fourier transform (STFT) to reduce the inherent algorithmic latency of RT-GCC-NMF from 64 ms to 2 ms with no loss in performance. We demonstrate that latencies within the tolerable range for hearing aids are possible on current hardware platforms.

show abstract

Speech Intelligibility Potential of General and Specialized Deep Neural Network Based Speech Enhancement Systems

Cited by 187 publications

References 53 publications

On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

rVAD: An unsupervised segment-based robust voice activity detection method

Unsupervised Low Latency Speech Enhancement With RT-GCC-NMF

Contact Info

Product

Resources

About