Coherent modification of pitch and energy for expressive prosody implantation

Sorin, Alexander; Shechtman, Slava; Pollet, Vincent

doi:10.1109/icassp.2015.7178905

“…In [52], consecutive spectral frames are used for duration modeling. Studies have also suggested some ways to perform intensity conversion based on F0 to improve naturalness [32].…”

Section: Introductionmentioning

confidence: 99%

High quality voice conversion using prosodic and high-resolution spectral features

Nguyen,

Lee,

Tian

et al. 2015

Preprint

0

View full text Add to dashboard Cite

Voice conversion methods have advanced rapidly over the last decade. Studies have shown that speaker characteristics are captured by spectral feature as well as various prosodic features. Most existing conversion methods focus on the spectral feature as it directly represents the timbre characteristics, while some conversion methods have focused only on the prosodic feature represented by the fundamental frequency. In this paper, a comprehensive framework using deep neural networks to convert both timbre and prosodic features is proposed. The timbre feature is represented by a high-resolution spectral feature. The prosodic features include F0, intensity and duration. It is well known that DNN is useful as a tool to model highdimensional features. In this work, we show that DNN initialized by our proposed autoencoder pretraining yields good quality DNN conversion models. This pretraining is tailor-made for voice conversion and leverages on autoencoder to capture the generic spectral shape of source speech. Additionally, our framework uses segmental DNN models to capture the evolution of the prosodic features over time. To reconstruct the converted speech, the spectral feature produced by the DNN model is combined with the three prosodic features produced by the DNN segmental models. Our experimental results show that the application of both prosodic and high-resolution spectral features leads to quality converted speech as measured by objective evaluation and subjective listening tests.

show abstract

“…In [52], consecutive spectral frames are used for duration modeling. Studies have also suggested some ways to perform intensity conversion based on F0 to improve naturalness [32].…”

Section: Introductionmentioning

confidence: 99%

High quality voice conversion using prosodic and high-resolution spectral features

Nguyen

¹

,

Lee

²

,

Tian

³

et al. 2015

Multimed Tools Appl

View full text Add to dashboard Cite

Voice conversion methods have advanced rapidly over the last decade. Studies have shown that speaker characteristics are captured by spectral feature as well as various prosodic features. Most existing conversion methods focus on the spectral feature as it directly represents the timbre characteristics, while some conversion methods have focused only on the prosodic feature represented by the fundamental frequency. In this paper, a comprehensive framework using deep neural networks to convert both timbre and prosodic features is proposed. The timbre feature is represented by a high-resolution spectral feature. The prosodic features include F0, intensity and duration. It is well known that DNN is useful as a tool to model highdimensional features. In this work, we show that DNN initialized by our proposed autoencoder pretraining yields good quality DNN conversion models. This pretraining is tailor-made for voice conversion and leverages on autoencoder to capture the generic spectral shape of source speech. Additionally, our framework uses segmental DNN models to capture the evolution of the prosodic features over time. To reconstruct the converted speech, the spectral feature produced by the DNN model is combined with the three prosodic features produced by the DNN segmental models. Our experimental results show that the application of both prosodic and high-resolution spectral features leads to quality converted speech as measured by objective evaluation and subjective listening tests.

show abstract

“…In [32], consecutive spectral frames are used for duration modeling. Studies have also suggested some ways to perform intensity conversion based on F0 to improve naturalness [34].…”

Section: Prosody Transformationmentioning

confidence: 99%

“…Among these, [30] and [47] have focused on wavelet domain instead of on the frequency domain directly. The most popular method is the mean-variance global transformation, adjusting the average level and the range of source F0 values [34]. This method retains the general an output vector is generated which represents the converted speech parameters.…”

Section: The Comprehensive Framework: Spectrum F0 and Intensity And mentioning

confidence: 99%

Voice conversion using deep neural networks

Nguyen¹

1

0

View full text Add to dashboard Cite

This thesis focuses on techniques to improve the performance of voice conversion. Voice conversion modifies the recorded speech of a source speaker towards a given target speaker. The resultant speech is to sound like the target speaker with the language content unchanged. This technology has been applied to create personalized voice in text-to-speech or virtual avatar, speech-to-singing synthesis or spoofing attacks in speaker verification systems.

show abstract

Coherent modification of pitch and energy for expressive prosody implantation

Cited by 5 publications

References 10 publications

High quality voice conversion using prosodic and high-resolution spectral features

High quality voice conversion using prosodic and high-resolution spectral features

High quality voice conversion using prosodic and high-resolution spectral features

Voice conversion using deep neural networks

Contact Info

Product

Resources

About