Speech Denoising by Parametric Resynthesis

Abstract

This work proposes the use of clean speech vocoder parameters as the target for a neural network performing speech enhancement. These parameters have been designed for text-tospeech synthesis so that they both produce high-quality resyntheses and also are straightforward to model with neural networks, but have not been utilized in speech enhancement until now. In comparison to a matched text-to-speech system that is given the ground truth transcripts of the noisy speech, our model is able to produce more natural speech because it has access to the true prosody in the noisy speech. In comparison to two denoising systems, the oracle Wiener mask and a DNNbased mask predictor, our model equals the oracle Wiener mask in subjective quality and intelligibility and surpasses the realistic system. A vocoder-based upper bound shows that there is still room for improvement with this approach beyond the oracle Wiener mask. We test speaker-dependence with two speakers and show that a single model can be used for multiple speakers.

Parametric Resynthesis With Neural Vocoders

“…We compare these two PR-neural models with PR-World, our previously proposed model [2], where the WORLD vocoder is used and the intermediate acoustic parameters are the fundamendal frequency, spectral envelope, and band aperiodicity used by WORLD [3]. Note that WORLD does not support 22 kHz sampling rates, so this system generates output at 16 kHz.…”

“…where J is the number of coupling transformations, K is the number of convolutions, log P (z) is the log-likelihood of the spherical Gaussian with variance ν 2 and in training ν = 1 is used. Note that WaveGlow refers to this parameter as σ, but we use ν to avoid confusion with the logistic function in (2). We use the official published waveGlow implementation 2 with original setup (12 coupling layers, each consisting of 8 layers of dilated convolution with 512 residual and 256 skip connections).…”

“…In our previous work [2], we built a PR system with a nonneural vocoder, WORLD [3]. Compared to such non-neural vocoders, neural vocoders like WaveNet [4] synthesize higher quality speech, as shown in the speech synthesis literature [4][5][6][7][8][9].…”

Speaker Independence of Neural Vocoders and Their Effect on Parametric Resynthesis Speech Enhancement

“…Parametric Resynthesis (PR) systems [2,3] predict clean acoustic parameters from noisy speech and synthesize speech from these predicted parameters using a speech synthesizer or vocoder. Current speech synthesizers are trained to generate high quality speech for a single speaker.…”

“…Current speech synthesizers are trained to generate high quality speech for a single speaker. In previous work we showed that a single speaker PR system can synthesize very high quality clean speech at 22 KHz [2] and performs better than the corresponding TTS system [3]. Hence, a critical question is whether these systems can be generalized to unknown speakers.…”