SINGAN: Singing Voice Conversion with Generative Adversarial Networks

Şişman, Berrak; Vijayan, Karthika; Dong, Minghui; Li, Haizhou

doi:10.1109/apsipaasc47483.2019.9023162

Cited by 38 publications

(24 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These approaches model parallel training samples using statistical methods, such as Gaussian mixture model (GMM)-based many-tomany eigenvoice conversion [11], direct waveform modification based on spectrum difference [12,13]. GAN-based parallel approach has also been proposed to improve conversion performance [14]. Since parallel SVC approaches require parallel data, which is expensive to collect, for the training process, researchers have investigated many non-parallel SVC approaches.…”

Section: Singing Voice Conversionmentioning

confidence: 99%

DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion

Liu¹,

Cao²,

Su³

et al. 2021

Preprint

View full text Add to dashboard Cite

Singing voice conversion (SVC) is one promising technique which can enrich the way of human-computer interaction by endowing a computer the ability to produce high-fidelity and expressive singing voice. In this paper, we propose DiffSVC, an SVC system based on denoising diffusion probabilistic model. DiffSVC uses phonetic posteriorgrams (PPGs) as content features. A denoising module is trained in DiffSVC, which takes destroyed mel spectrogram produced by the diffusion/forward process and its corresponding step information as input to predict the added Gaussian noise. We use PPGs, fundamental frequency features and loudness features as auxiliary input to assist the denoising process. Experiments show that DiffSVC can achieve superior conversion performance in terms of naturalness and voice similarity to current state-of-the-art SVC approaches.

show abstract

Section: Singing Voice Conversionmentioning

confidence: 99%

DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion

Liu¹,

Cao²,

Su³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The early studies for singing voice conversion generally follow the statistical generation architectures [1,2], which often use Gaussian mixture model (GMM) with parallel singing data. [3] updates the conversion framework with Generative Adversarial Network (GAN). However, it still requires the source and the target speakers to sing the same songs during the training phase.…”

Section: Introductionmentioning

confidence: 99%

PPG-Based Singing Voice Conversion with Adversarial Representation Learning

Tang

Yin

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Singing voice conversion (SVC) aims to convert the voice of one singer to that of other singers while keeping the singing content and melody. On top of recent voice conversion works, we propose a novel model to steadily convert songs while keeping their naturalness and intonation. We build an endto-end architecture, taking phonetic posteriorgrams (PPGs) as inputs and generating mel spectrograms. Specifically, we implement two separate encoders: one encodes PPGs as content, and the other compresses mel spectrograms to supply acoustic and musical information. To improve the performance on timbre and melody, an adversarial singer confusion module and a mel-regressive representation learning module are designed for the model. Objective and subjective experiments are conducted on our private Chinese singing corpus. Comparing with the baselines, our methods can significantly improve the conversion performance in terms of naturalness, melody, and voice similarity. Moreover, our PPG-based method is proved to be robust for noisy sources.

show abstract

“…statistical methods, such as Gaussian mixture model (GMM) based many-to-many eigenvoice conversion [1], direct waveform modification based on spectrum difference [2,3]. Artificial neural network (ANN) based approaches are also proposed to improve conversion performance [4,5].…”

Section: Introductionmentioning

confidence: 99%

Fastsvc: Fast Cross-Domain Singing Voice Conversion With Feature-Wise Linear Modulation

Liu

Cao

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

This paper presents FastSVC, a light-weight cross-domain singing voice conversion (SVC) system, which can achieve high conversion performance, with inference speed 4x faster than real-time on CPUs. FastSVC uses Conformer-based phoneme recognizer to extract singer-agnostic linguistic features from singing signals. A feature-wise linear modulation based generator is used to synthesize waveform directly from linguistic features, leveraging information from sineexcitation signals and loudness features. The waveform generator can be trained conveniently using a multi-resolution spectral loss and an adversarial loss. Experimental results show that the proposed FastSVC system, compared with a computationally heavy baseline system, can achieve comparable conversion performance in some scenarios and significantly better conversion performance in other scenarios. Moreover, the proposed FastSVC system achieves desirable cross-lingual singing conversion performance. The inference speed of the FastSVC system is 3x and 70x faster than the baseline system on GPUs and CPUs, respectively.

show abstract

SINGAN: Singing Voice Conversion with Generative Adversarial Networks

Cited by 38 publications

References 31 publications

DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion

DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion

PPG-Based Singing Voice Conversion with Adversarial Representation Learning

Fastsvc: Fast Cross-Domain Singing Voice Conversion With Feature-Wise Linear Modulation

Contact Info

Product

Resources

About