StarGAN-VC: non-parallel many-to-many Voice Conversion Using Star Generative Adversarial Networks

Abstract

We have previously proposed a method that allows for non-parallel voice conversion (VC) by using a variant of generative adversarial networks (GANs) called StarGAN. The main features of our method, called StarGAN-VC, are as follows: First, it requires no parallel utterances, transcriptions, or time alignment procedures for speech generator training. Second, it can simultaneously learn mappings across multiple domains using a single generator network so that it can fully exploit available training data collected from multiple domains to capture latent features that are common to all the domains. Third, it is able to generate converted speech signals quickly enough to allow real-time implementations and requires only several minutes of training examples to generate reasonably realistic-sounding speech. In this paper, we describe three formulations of StarGAN, including a newly introduced novel StarGAN variant called "Augmented classifier StarGAN (A-StarGAN)", and compare them in a non-parallel VC task. We also compare them with several baseline methods.

Many-To-Many Voice Conversion Using Conditional Cycle-Consistent Adversarial Networks

“…We compared the proposed CC-GAN based method with two conventional approaches, i.e., the CycleGAN based VC [10] and the StarGAN based VC [18], using the VCC2018 corpus [20]. The speakers in the corpus consist of 6 males and 6 females.…”

“…'H', 'W', 'C', 'K', and 'S' represent height, width, number of channels, kernel size, and stride, respectively. [18] for the comparison as it was impractical to train all 66 CycleGAN models due to the time and memory constraints. On the other hand, since we were not able to reproduce the same results as in [18] for the StarGAN based VC, we downloaded the converted speech samples from the author's website and used them for the comparative evaluations.…”

“…Though the number of sample utterances is small, we think this is fair comparison. We also trained a 4-speaker CC-GAN model for the fair comparison with [18]. We conducted objective evaluations as well as subjective evaluations.…”

“…The idea of many-to-many voice conversion by extending the CycleGAN first appeared in [14,18]. It was shown experimentally that the model size can be reduced by half for a two-speaker voice conversion case [14].…”

“…In the next section, we show how the CC-GAN for multi-speaker voice conversion performs when the number of speakers is more than two. Another extension of the CycleGAN for many-tomany voice conversion is the StarGAN [17] based voice conversion [18,19]. It is similar to the CC-GAN based manyto-many voice conversion in that the GAN is conditioned on the speaker identity vectors.…”