Singing Voice Synthesis Based on Generative Adversarial Networks

Hono, Yukiya; Hashimoto, Kei; Oura, Keiichiro; Nankaku, Yoshihiko; Tokuda, Keiichi

doi:10.1109/icassp.2019.8683154

Cited by 57 publications

(39 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non-Seq2Seq singing synthesizers include those based on autoregressive architectures [17,21,22], feed-forward CNN [23], and feed-forward GAN-based approaches [24,25].…”

Section: Relation To Prior Workmentioning

confidence: 99%

Sequence-to-Sequence Singing Synthesis Using the Feed-Forward Transformer

Blaauw

Bonada

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We propose a sequence-to-sequence singing synthesizer, which avoids the need for training data with pre-aligned phonetic and acoustic features. Rather than the more common approach of a content-based attention mechanism combined with an autoregressive decoder, we use a different mechanism suitable for feed-forward synthesis. Given that phonetic timings in singing are highly constrained by the musical score, we derive an approximate initial alignment with the help of a simple duration model. Then, using a decoder based on a feed-forward variant of the Transformer model, a series of self-attention and convolutional layers refines the result of the initial alignment to reach the target acoustic features. Advantages of this approach include faster inference and avoiding the exposure bias issues that affect autoregressive models trained by teacher forcing. We evaluate the effectiveness of this model compared to an autoregressive baseline, the importance of self-attention, and the importance of the accuracy of the duration model.

show abstract

“…Non-Seq2Seq singing synthesizers include those based on autoregressive architectures [17,21,22], feed-forward CNN [23], and feed-forward GAN-based approaches [24,25].…”

Section: Relation To Prior Workmentioning

confidence: 99%

Sequence-to-Sequence Singing Synthesis Using the Feed-Forward Transformer

Blaauw

Bonada

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…(6) are fed to the discriminator. This method is generally called conditional GAN [19] and its effectiveness has been shown in SPSS [15,16]. Figure 1: Outline of multi-task learning with speaker identification.…”

Section: Speaker Information Utilization Via Conditional Ganmentioning

confidence: 99%

“…Toda et al [10] focused on the significant decrease in the synthetic acoustic feature's global variance (GV) and gave a constraint at the parameter generation so as to generate acoustic features with appropriate GV. In contrast, generative adversarial networks (GANs) [13] have attempted to decrease the difference in acoustic features between synthesized and real speech [14,15,16]. They focus on the remarkable difference in acoustic feature distribution between natural and a synthetic speech [17].…”

Section: Introductionmentioning

confidence: 99%

Multi-Speaker Modeling for DNN-based Speech Synthesis Incorporating Generative Adversarial Networks

Kanagawa¹,

Ijima²

2019

10th ISCA Workshop on Speech Synthesis (SSW 10)

View full text Add to dashboard Cite

This paper presents a novel DNN-based speech synthesis method we derived from multi-speaker training data. In general, speaker-dependent modeling techniques based on generative adversarial networks (GANs) improve synthesized speech quality. However, they are inadequate for multi-speaker training because conventional discriminators cannot take into account speaker identity, which degrades anti-spoofing performance in GAN discriminators. We introduce two approaches as means to learn GAN speaker characteristics, i.e., auxiliary features and tasks. The first uses speaker codes as additional discriminator input. The second uses speaker identification as a means to verify that anti-spoofing verification methods are effective. Experimental results showed that our proposed techniques outperformed conventional and GAN-based methods.

show abstract

“…Another approach for parallel waveform generation is to use generative adversarial networks (GANs) [12]. A GAN is a powerful generative model that has been successfully used in various research fields such as image generation [13], speech synthesis [14], and singing voice synthesis [15]. GAN-based models have also been proposed for waveform generation [16,17].…”

Section: Introductionmentioning

confidence: 99%

Periodnet: A Non-Autoregressive Waveform Generation Model with a Structure Separating Periodic and Aperiodic Components

Hono

Takaki

Hashimoto

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

We propose PeriodNet, a non-autoregressive (non-AR) waveform generation model with a new model structure for modeling periodic and aperiodic components in speech waveforms. The non-AR waveform generation models can generate speech waveforms parallelly and can be used as a speech vocoder by conditioning an acoustic feature. Since a speech waveform contains periodic and aperiodic components, both components should be appropriately modeled to generate a high-quality speech waveform. However, it is difficult to decompose the components from a natural speech waveform in advance. To address this issue, we propose a parallel model and a series model structure separating periodic and aperiodic components. The features of our proposed models are that explicit periodic and aperiodic signals are taken as input, and external periodic/aperiodic decomposition is not needed in training. Experiments using a singing voice corpus show that our proposed structure improves the naturalness of the generated waveform. We also show that the speech waveforms with a pitch outside of the training data range can be generated with more naturalness.

show abstract

Singing Voice Synthesis Based on Generative Adversarial Networks

Cited by 57 publications

References 12 publications

Sequence-to-Sequence Singing Synthesis Using the Feed-Forward Transformer

Sequence-to-Sequence Singing Synthesis Using the Feed-Forward Transformer

Multi-Speaker Modeling for DNN-based Speech Synthesis Incorporating Generative Adversarial Networks

Periodnet: A Non-Autoregressive Waveform Generation Model with a Structure Separating Periodic and Aperiodic Components

Contact Info

Product

Resources

About