VocGAN: A High-Fidelity Real-Time Vocoder with a Hierarchically-Nested Adversarial Network

Yang, Jinhyeok; Lee, Junmo; Kim, Young-Ik; Cho, Hoon Young; Kim, In Jung

doi:10.21437/interspeech.2020-1238

Cited by 54 publications

(44 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GAN-TTS also adopts an ensemble of 10 similar discriminators with different input window sizes with or without the conditional acoustic features to guide its generator to learn different aspects of speech information. Furthermore, the variants of MelGAN such as VocGAN [28] adopted a multi-scale generator and a hierarchically-nested discriminator and multiband MelGAN [29] incorporated a multi-band technique into MelGAN also achieved further speech quality or generative efficiency improvements.…”

Section: Natural Speechmentioning

confidence: 99%

Quasi-Periodic Parallel WaveGAN: A Non-Autoregressive Raw Waveform Generative Model With Pitch-Dependent Dilated Convolution Neural Network

Hayashi

Okamoto

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

In this paper, we propose a quasi-periodic parallel WaveGAN (QPPWG) waveform generative model, which applies a quasi-periodic (QP) structure to a parallel WaveGAN (PWG) model using pitch-dependent dilated convolution networks (PD-CNNs). PWG is a small-footprint GAN-based raw waveform generative model, whose generation time is much faster than real time because of its compact model and non-autoregressive (non-AR) and non-causal mechanisms. Although PWG achieves high-fidelity speech generation, the generic and simple network architecture lacks pitch controllability for an unseen auxiliary fundamental frequency (F0) feature such as a scaled F0. To improve the pitch controllability and speech modeling capability, we apply a QP structure with PDCNNs to PWG, which introduces pitch information to the network by dynamically changing the network architecture corresponding to the auxiliary F0 feature. Both objective and subjective experimental results show that QP-PWG outperforms PWG when the auxiliary F0 feature is scaled. Moreover, analyses of the intermediate outputs of QPPWG also show better tractability and interpretability of QPPWG, which respectively models spectral and excitation-like signals using the cascaded fixed and adaptive blocks of the QP structure. Index Terms-Neural vocoder, parallel WaveGAN, quasiperiodic WaveNet, pitch-dependent dilated convolution I. INTRODUCTION S PEECH generation is a technique to generate specific speech according to given inputs such as texts (text-tospeech, TTS), the speech of a source speaker (speaker voice conversion, VC), and noisy speech (speech enhancement, SE). The core of speech generation is the controllability of speech components, and the fundamental technique is called a vocoder [1]-[3]. A vocoder encodes speech into acoustic Manuscript received xxx xx, 2020; revised xxx xx, 2020.

show abstract

Section: Natural Speechmentioning

confidence: 99%

Quasi-Periodic Parallel WaveGAN: A Non-Autoregressive Raw Waveform Generative Model With Pitch-Dependent Dilated Convolution Neural Network

Hayashi

Okamoto

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Similar ideas are used in Multiband-MelGAN [15], which generates each sub-band of the target speech separately, saving computational power, and then obtains the final waveform using a synthesis PQMF. Research in this field is very active and we can cite the very recent GAN vocoders such as VocGan [16] and HiFi-GAN [17].…”

Section: Related Workmentioning

confidence: 99%

StyleMelGAN: An Efficient High-Fidelity Adversarial Vocoder with Temporal Adaptive Normalization

Mustafa

Pia

Fuchs

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In recent years, neural vocoders have surpassed classical speech generation approaches in naturalness and perceptual quality of the synthesized speech. Computationally heavy models like WaveNet and WaveGlow achieve best results, while lightweight GAN models, e.g. MelGAN and Parallel WaveGAN, remain inferior in terms of perceptual quality. We therefore propose StyleMelGAN, a lightweight neural vocoder allowing synthesis of high-fidelity speech with low computational complexity. StyleMelGAN employs temporal adaptive normalization to style a low-dimensional noise vector with the acoustic features of the target speech. For efficient training, multiple random-window discriminators adversarially evaluate the speech signal analyzed by a filter bank, with regularization provided by a multi-scale spectral reconstruction loss. The highly parallelizable speech generation is several times faster than real-time on CPUs and GPUs. MUSHRA and P.800 listening tests show that StyleMelGAN outperforms prior neural vocoders in copy-synthesis and Text-to-Speech scenarios.

show abstract

“…HiFi-GAN [19] consists of small sub-discriminators obtaining specific periodic parts of raw waveforms, achieving higher computational efficiency and sample quality. VocGAN [41] applies the joint conditional and unconditional objective, which is inspired by successful results in high-resolution image synthesis. Although these vocoders could be applied in SVS systems, distinct degradations occur when generalizing those systems to unseen singers.…”

Section: Vocodermentioning

confidence: 99%

Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus

Huang

Chen

Ren

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

High-fidelity multi-singer singing voice synthesis is challenging for neural vocoder due to the singing voice data shortage, limited singer generalization, and large computational cost. Existing open corpora could not meet requirements for high-fidelity singing voice synthesis because of the scale and quality weaknesses. Previous vocoders have difficulty in multi-singer modeling, and a distinct degradation emerges when conducting unseen singer singing voice generation. To accelerate singing voice researches in the community, we release a large-scale, multi-singer Chinese singing voice dataset OpenSinger. To tackle the difficulty in unseen singer modeling, we propose Multi-Singer, a fast multi-singer vocoder with generative adversarial networks. Specifically, 1) Multi-Singer uses a multi-band generator to speed up both training and inference procedure. 2) to capture and rebuild singer identity from the acoustic feature (i.e., mel-spectrogram), Multi-Singer adopts a singer conditional discriminator and conditional adversarial training objective.3) to supervise the reconstruction of singer identity in the spectrum envelopes in frequency domain, we propose an auxiliary singer perceptual loss. The joint training approach effectively works in GANs for multi-singer voices modeling. Experimental results verify the effectiveness of OpenSinger and show that Multi-Singer improves unseen singer singing voices modeling in both speed and quality over previous methods. The further experiment proves that combined with FastSpeech 2 as the acoustic model, Multi-Singer achieves strong robustness in the multi-singer singing voice synthesis pipeline. Samples are available at https://Multi-Singer.github.io/ CCS CONCEPTS• Applied computing → Sound and music computing; • Computing methodologies → Natural language generation.

show abstract

VocGAN: A High-Fidelity Real-Time Vocoder with a Hierarchically-Nested Adversarial Network

Cited by 54 publications

References 12 publications

Quasi-Periodic Parallel WaveGAN: A Non-Autoregressive Raw Waveform Generative Model With Pitch-Dependent Dilated Convolution Neural Network

Quasi-Periodic Parallel WaveGAN: A Non-Autoregressive Raw Waveform Generative Model With Pitch-Dependent Dilated Convolution Neural Network

StyleMelGAN: An Efficient High-Fidelity Adversarial Vocoder with Temporal Adaptive Normalization

Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus

Contact Info

Product

Resources

About