Speech Bandwidth Extension Using Generative Adversarial Networks

Li, Sen; Villette, S.; Ramadas, Pravin; Sinder, Daniel J.

doi:10.1109/icassp.2018.8462588

Cited by 42 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For BWE, its variable discrimination helps to refine details in the high frequencies. The previous GAN works in BWE [31,20,19] typically follow simple designs, using a discriminator of a few fully connected layers or convolutional layers on the spectral features, while the discrimination directly on waveform has rarely been employed.…”

Section: Related Workmentioning

confidence: 99%

Bandwidth Extension is All You Need

Wang

Finkelstein

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Speech generation and enhancement have seen recent breakthroughs in quality thanks to deep learning. These methods typically operate at a limited sampling rate of 16-22kHz due to computational complexity and available datasets. This limitation imposes a gap between the output of such methods and that of high-fidelity (≥44kHz) real-world audio applications. This paper proposes a new bandwidth extension (BWE) method that expands 8-16kHz speech signals to 48kHz. The method is based on a feed-forward WaveNet architecture trained with a GAN-based deep feature loss. A mean-opinionscore (MOS) experiment shows significant improvement in quality over state-of-the-art BWE methods. An AB test reveals that our 16to-48kHz BWE is able to achieve fidelity that is typically indistinguishable from real high-fidelity recordings. We use our method to enhance the output of recent speech generation and denoising methods, and experiments demonstrate significant improvement in sound quality over these baselines. We propose this as a general approach to narrow the gap between generated speech and recorded speech, without the need to adapt such methods to higher sampling rates.

show abstract

Section: Related Workmentioning

confidence: 99%

Bandwidth Extension is All You Need

Wang

Finkelstein

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Generative Adversarial Networks (GANs) [1] are a class of approaches for learning generative models based on game theory. They find various applications in image processing [2,3,4], wireless communications [5,6] and signal processing [7,8]. The GANs framework can be also easily modified with other loss functions and learning dynamics, triggering numerous variants [9,10,11,12,13,14,15].…”

Section: Introductionmentioning

confidence: 99%

“…The value inside the bracket is default choice in our experiment.3 https://github.com/znxlwm/pytorch-generative-modelcollections4 http://yann.lecun.com/exdb/mnist/ 5 https://github.com/zalandoresearch/fashion-mnist/ 6 https://www.cs.toronto.edu/∼kriz/cifar.html7 We use a small version of imagenet publicly available online: http://image-net.org/small/train_64x64.tar…”

mentioning

confidence: 99%

Relaxed Wasserstein with Applications to GANs

Guo

Hong

Lin

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Wasserstein Generative Adversarial Networks (WGANs) provide a versatile class of models, which have attracted great attention in various applications. However, this framework has two main drawbacks: (i) Wasserstein-1 (or Earth-Mover) distance is restrictive such that WGANs cannot always fit data geometry well; (ii) It is difficult to achieve fast training of WGANs. In this paper, we propose a new class of Relaxed Wasserstein (RW) distances by generalizing Wasserstein-1 distance with Bregman cost functions. We show that RW distances achieve nice statistical properties while not sacrificing the computational tractability. Combined with the GANs framework, we develop Relaxed WGANs (RWGANs) which are not only statistically flexible but can be approximated efficiently using heuristic approaches. Experiments on real images demonstrate that the RWGAN with Kullback-Leibler (KL) cost function outperforms other competing approaches, e.g., WGANs, even with gradient penalty.

show abstract

“…GANs have predominantly been used in computer vision, including but not limited to image generation, face synthesis [8], image translation [9,10,11], texture synthesis [12,13], medical imaging, [14] and super-resolution [15]. Moreover, GANs can be applied in many other fields including but not limited to voice and speech signals [16,17,18], anomaly detection [19], power systems and smart grids [20,21,22], electronics [23,24], and fault diagnosis [25,26,27,28].…”

Section: Introductionmentioning

confidence: 99%