DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion

Liu, Songxiang; Cao, Yuewen; Su, Dan; Meng, Helen

doi:10.48550/arxiv.2105.13871

Cited by 3 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model is trained by optimizing the evidence lower bound (ELBO) during the diffusion process. Recently, the diffusion probabilistic models have been shown to provide outstanding performance in generative modeling for natural images [30], [31], and raw audio waveforms [32], [33]. As reported in [32], the DiffWave model, formed by the diffusion probabilistic model, can yield state-of-the-art performance on either conditional or unconditional waveform generation tasks with a small number of parameters.…”

Section: Introductionmentioning

confidence: 99%

A Study on Speech Enhancement Based on Diffusion Probabilistic Model

Lu¹,

Tsao²,

Watanabe³

2021

Preprint

View full text Add to dashboard Cite

Diffusion probabilistic models have demonstrated an outstanding capability to model natural images and raw audio waveforms through a paired diffusion and reverse processes. The unique property of the reverse process (namely, eliminating non-target signals from the Gaussian noise and noisy signals) could be utilized to restore clean signals. Based on this property, we propose a diffusion probabilistic model-based speech enhancement (DiffuSE) model that aims to recover clean speech signals from noisy signals. The fundamental architecture of the proposed DiffuSE model is similar to that of DiffWave-a highquality audio waveform generation model that has a relatively low computational cost and footprint. To attain better enhancement performance, we designed an advanced reverse process, termed the supportive reverse process, which adds noisy speech in each time-step to the predicted speech. The experimental results show that DiffuSE yields performance that is comparable to related audio generative models on the standardized Voice Bank corpus SE task. Moreover, relative to the generally suggested full sampling schedule, the proposed supportive reverse process especially improved the fast sampling, taking few steps to yield better enhancement results over the conventional full step inference process.

show abstract

Section: Introductionmentioning

confidence: 99%

A Study on Speech Enhancement Based on Diffusion Probabilistic Model

Lu¹,

Tsao²,

Watanabe³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Diffusion (score-based) models, which are originated from non-equilibrium statistical physics, have recently shown impressive successes on sample generations of a wide range of types, including images [10,21,38,50] , 3D point clouds [13,35] and audio generation [31,34], among others. In addition to concrete applications of various diffusion generative models, it is also desirable to analyze them in an appropriate and flexible framework, by which novel improvements can be further developed.…”

Section: Introductionmentioning

confidence: 99%

A Flexible Diffusion Model

Wei¹,

Yang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Diffusion (score-based) generative models have been widely used for modeling various types of complex data, including images, audios, and point clouds. Recently, the deep connection between forward-backward stochastic differential equations (SDEs) and diffusion-based models has been revealed, and several new variants of SDEs are proposed (e.g., sub-VP, critically-damped Langevin) along this line. Despite the empirical success of the hand-crafted fixed forward SDEs, a great quantity of proper forward SDEs remain unexplored. In this work, we propose a general framework for parameterizing the diffusion model, especially the spatial part of the forward SDE. An abstract formalism is introduced with theoretical guarantees, and its connection with previous diffusion models is leveraged. We demonstrate the theoretical advantage of our method from an optimization perspective. Numerical experiments on synthetic datasets, MINIST and CIFAR10 are also presented to validate the effectiveness of our framework.Preprint. Under review.

show abstract

Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme

Попов

Vovk

Gogoryan

et al. 2021

Preprint

View full text Add to dashboard Cite

Voice conversion is a common speech synthesis task which can be solved in different ways depending on a particular real-world scenario. The most challenging one often referred to as one-shot many-to-many voice conversion consists in copying the target voice from only one reference utterance in the most general case when both source and target speakers do not belong to the training dataset. We present a scalable high-quality solution based on diffusion probabilistic modeling and demonstrate its superior quality compared to state-of-the-art one-shot voice conversion approaches. Moreover, focusing on real-time applications, we investigate general principles which can make diffusion models faster while keeping synthesis quality at a high level. As a result, we develop a novel Stochastic Differential Equations solver suitable for various diffusion model types and generative tasks as shown through empirical studies and justify it by theoretical analysis.

show abstract

DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion

Cited by 3 publications

References 19 publications

A Study on Speech Enhancement Based on Diffusion Probabilistic Model

A Study on Speech Enhancement Based on Diffusion Probabilistic Model

A Flexible Diffusion Model

Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme

Contact Info

Product

Resources

About