HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks

Su, Jiaqi; Jin, Zeyu; Finkelstein, Adam

doi:10.48550/arxiv.2006.05694

Cited by 18 publications

(21 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An asterisk means that the difference between ModDW and DW is statistically significant with p < 0.05. (3,8), stride size is (1,4), and the channel size is kept the same as the input. For the 2-D CNN, the settings are the same and the channel size is 64, 16, 8, 4, 1.…”

Section: Deep Cnn For Conditioner Upsamplingmentioning

confidence: 99%

“…Speech enhancement (SE) of degraded speech is important across many applications including telecommunications [1], speech recognition [2], etc. Many methods have been developed for similar applications, such as speech denoising, dereverberation and equalization [3,4]. Most current SE methods are designed to remove background noise, in many cases using an additive noise model.…”

Section: Introductionmentioning

confidence: 99%

“…While these methods work well for additive noise conditions, they are not suitable for our applica-tion. Enhancement methods based on machine learning models such as diffusion models and U-nets with adversarial loss have resulted in a sizeable improvement in performance [1,3,4,8]. These models can enhance speech quality, however they require complex network structures with a large number of parameters.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Restoring degraded speech via a modified diffusion model

Zhang,

Jayasuriya,

Berisha

2021

Preprint

View full text Add to dashboard Cite

There are many deterministic mathematical operations (e.g. compression, clipping, downsampling) that degrade speech quality considerably. In this paper we introduce a neural network architecture, based on a modification of the DiffWave model, that aims to restore the original speech signal. Dif-fWave, a recently published diffusion-based vocoder, has shown state-of-the-art synthesized speech quality and relatively shorter waveform generation times, with only a small set of parameters. We replace the mel-spectrum upsampler in DiffWave with a deep CNN upsampler, which is trained to alter the degraded speech mel-spectrum to match that of the original speech. The model is trained using the original speech waveform, but conditioned on the degraded speech mel-spectrum. Post-training, only the degraded mel-spectrum is used as input and the model generates an estimate of the original speech. Our model results in improved speech quality (original DiffWave model as baseline) on several different experiments. These include improving the quality of speech degraded by LPC-10 compression, AMR-NB compression, and signal clipping. Compared to the original DiffWave architecture, our scheme achieves better performance on several objective perceptual metrics and in subjective comparisons. Improvements over baseline are further amplified in a out-of-corpus evaluation setting.

show abstract

Section: Deep Cnn For Conditioner Upsamplingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Restoring degraded speech via a modified diffusion model

Zhang,

Jayasuriya,

Berisha

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…VoCo [65] Dereverb [66] HiFi-GAN [67] FFTnet [ precision@K over all test queries turns out to be 0.97 for K = 10 and 0.95 for K = 25. These high precision retrievals show that the embeddings indeed capture quality.…”

Section: Objective Evaluationsmentioning

confidence: 99%

“…We consider an exhaustive set of 10 different datasets for this evaluation. These datasets span over a variety of well-known speech problems; (1) Speech Synthesis (VoCo [65] and FFTnet [68]), (2) Speech Enhancement (Dereverberation [66], Noizeus [71], HiFi-GAN [67]), (3) Voice Conversion (VCC-2018 [70]), (4) Speech Source Separation (PEASS [69]), (5) Telephony Degradations [72], (6) Bandwidth Extension (BWE [73]), and (7) General Degradation's (Simulated [6]). Please refer to supplementary material for details about these datasets.…”

Section: Subjective Evaluationsmentioning

confidence: 99%

NORESQA: A Framework for Speech Quality Assessment using Non-Matching References

Manocha,

Xu,

Kumar

2021

Preprint

View full text Add to dashboard Cite

The perceptual task of speech quality assessment (SQA) is a challenging task for machines to do. Objective SQA methods that rely on the availability of the corresponding clean reference have been the primary go-to approaches for SQA. Clearly, these methods fail in real-world scenarios where the ground truth clean references are not available. In recent years, non-intrusive methods that train neural networks to predict ratings or scores have attracted much attention, but they suffer from several shortcomings such as lack of robustness, reliance on labeled data for training and so on. In this work, we propose a new direction for speech quality assessment. Inspired by human's innate ability to compare and assess the quality of speech signals even when they have non-matching contents, we propose a novel framework that predicts a subjective relative quality score for the given speech signal with respect to any provided reference without using any subjective data. We show that neural networks trained using our framework produce scores that correlate well with subjective mean opinion scores (MOS) and are also competitive to methods such as DNSMOS [1], which explicitly relies on MOS from humans for training networks. Moreover, our method also provides a natural way to embed quality-related information in neural networks, which we show is helpful for downstream tasks such as speech enhancement.Preprint. Under review.

show abstract