Speech Enhancement for Noise-Robust Speech Synthesis Using Wasserstein GAN

Abstract

The quality of speech synthesis systems can be significantly deteriorated by the presence of background noise in the recordings. Despite the existence of speech enhancement techniques for effectively suppressing additive noise under low signal-tonoise (SNR) conditions, these techniques have been neither designed nor tested in speech synthesis tasks where background noise has relatively lower energy. In this paper, we propose a speech enhancement technique based on generative adversarial networks (GANs) which acts as a preprocessing step of speech synthesis. Motivated by the speech enhancement generative adversarial network (SEGAN) approach and recent advances in deep learning, we propose to use Wasserstein GAN (WGAN) with gradient penalty and gated activation functions to the autoencoder network of SEGAN. We studied the impact of the proposed method on a data set consisting of 28 speakers and different noise types with 3 different SNR level. The effectiveness of the proposed method in the context of speech synthesis is demonstrated through the training of WaveNet vocoder. We compare our method against SEGAN. Both subjective and objective metrics confirm that the proposed speech enhancement approach outperforms SEGAN in terms of speech synthesis quality.

A Flow-Based Neural Network for Time Domain Speech Enhancement

“…The values for batch size (4,8,12), number of flow blocks (8,12,16) and the amount of samples grouped together as input (8,12,24) were selected in a hyper-parameter search. As a loss function for training, the model minimized the negative log-likelihood of the given data.…”

“…This approach has been extended multiple times, e.g. by making use of the Wasserstein distance [16] or by combining multiple generators to increase performance [17]. Others reported strong SE results working with GANs to estimate clean T-F spectrograms by implementing additional techniques like a mean squared error regularization [12] or optimizing the network directly with respect to a speech specific evaluation metric [13].…”

Time-domain Speech Enhancement with Generative Adversarial Learning

“…To address this problem, the Wasserstein distance [16][17][18] has been introduced to improve the conditional GAN loss, resulting in the Wasserstein GAN (WGAN) method that achieves better objective performance than SEGAN [15,19]. The WGAN method is further improved in [20] by employing metric evaluation in the conditional GAN loss, and leads to the Metric GAN method, which outperforms WGAN based methods for speech enhancement.…”

A Neural Text-to-Speech Model Utilizing Broadcast Data Mixed with Background Music

“…One approach to solve this issue is to preprocess speech data before using it for training the TTS model. [6] and [7] processed noisy speech data using speech enhancement techniques for noise-robust TTS systems. Speech enhancement research has mainly focused on removing other types of noise rather than BGM, but the primary removal target for broadcasted data is BGM.…”