Voice Conversion for Whispered Speech Synthesis

Cotescu, Marius; Drugman, Thomas; Huybrechts, Goeric; Lorenzo-Trueba, Jaime; Moinet, Alexis

doi:10.1109/lsp.2019.2961213

Cited by 27 publications

(8 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Parallel WaveGAN (PWG) [20] was used as the neural vocoder. We followed an open-source implementation 3 . The training data of PWG contained the audio recordings of all control speakers in UASpeech.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Towards Identity Preserving Normal to Dysarthric Voice Conversion

Huang¹,

Halpern²,

Violeta³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a voice conversion framework that converts normal speech into dysarthric speech while preserving the speaker identity. Such a framework is essential for (1) clinical decision making processes and alleviation of patient stress, (2) data augmentation for dysarthric speech recognition. This is an especially challenging task since the converted samples should capture the severity of dysarthric speech while being highly natural and possessing the speaker identity of the normal speaker. To this end, we adopted a two-stage framework, which consists of a sequence-to-sequence model and a nonparallel frame-wise model. Objective and subjective evaluations were conducted on the UASpeech dataset, and results showed that the method was able to yield reasonable naturalness and capture severity aspects of the pathological speech. On the other hand, the similarity to the normal source speaker's voice was limited and requires further improvements.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Neural voice conversion (VC) has substantially improved the naturalness of synthesized speech in a wide range of tasks, including read speech [1], emotional speech [2] and whispered speech [3]. However, pathological VC (and TTS too) is a largely unexplored area, which has several interesting applications.…”

Section: Introductionmentioning

confidence: 99%

Towards Identity Preserving Normal to Dysarthric Voice Conversion

Huang¹,

Halpern²,

Violeta³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Voice Conversion (VC) is the task of modifying an utterance from a source speaker to make it sound like it was uttered by a target speaker, while preserving the original linguistic content [1]. VC is a key component of many modern applications, including text-to-speech (TTS) [2], speech enhancement [3], and speaking assistance [4] systems. Due to its success in these fields, VC has been studied extensively in recent years [5].…”

Section: Introductionmentioning

confidence: 99%

Voicy: Zero-Shot Non-Parallel Voice Conversion in Noisy Reverberant Environments

Mottini¹,

Lorenzo-Trueba²,

Karlapati³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Voice Conversion (VC) is a technique that aims to transform the non-linguistic information of a source utterance to change the perceived identity of the speaker. While there is a rich literature on VC, most proposed methods are trained and evaluated on clean speech recordings. However, many acoustic environments are noisy and reverberant, severely restricting the applicability of popular VC methods to such scenarios. To address this limitation, we propose Voicy, a new VC framework particularly tailored for noisy speech. Our method, which is inspired by the de-noising auto-encoders framework, is comprised of four encoders (speaker, content, phonetic and acoustic-ASR) and one decoder. Importantly, Voicy is capable of performing non-parallel zero-shot VC, an important requirement for any VC system that needs to work on speakers not seen during training. We have validated our approach using a noisy reverberant version of the LibriSpeech dataset. Experimental results show that Voicy outperforms other tested VC techniques in terms of naturalness and target speaker similarity in noisy reverberant environments.

show abstract

“…Most existing works on whisper and Lombard speech synthesis analyze and generate these speaking styles separately. Because whisper speech lacks the vibration element of voiced speech, the fundamental frequency (f0) and spectral tilt can be modified in a source-filter model to convert normal speech into whisper speech [10,11]. Other methods [12,11] learn the mapping between normal and whisper acoustic features through voice conversion (VC) techniques.…”

Section: Introductionmentioning

confidence: 99%

“…Because whisper speech lacks the vibration element of voiced speech, the fundamental frequency (f0) and spectral tilt can be modified in a source-filter model to convert normal speech into whisper speech [10,11]. Other methods [12,11] learn the mapping between normal and whisper acoustic features through voice conversion (VC) techniques. However, parallel corpora containing both normal and whisper recordings are usually needed for training a VC model.…”

Section: Introductionmentioning

confidence: 99%

Whispered and Lombard Neural Speech Synthesis

Bleisch

Petkov

et al. 2021

Preprint

View full text Add to dashboard Cite

It is desirable for a text-to-speech system to take into account the environment where synthetic speech is presented, and provide appropriate context-dependent output to the user. In this paper, we present and compare various approaches for generating different speaking styles, namely, normal, Lombard, and whisper speech, using only limited data. The following systems are proposed and assessed: 1) Pre-training and fine-tuning a model for each style. 2) Lombard and whisper speech conversion through a signal processing based approach. 3) Multi-style generation using a single model based on a speaker verification model. Our mean opinion score and AB preference listening tests show that 1) we can generate high quality speech through the pre-training/fine-tuning approach for all speaking styles. 2) Although our speaker verification (SV) model is not explicitly trained to discriminate different speaking styles, and no Lombard and whisper voice is used for pretrain this system, SV model can be used as style encoder for generating different style embeddings as input for Tacotron system. We also show that the resulting synthetic Lombard speech has a significant positive impact on intelligibility gain.

show abstract

Voice Conversion for Whispered Speech Synthesis

Cited by 27 publications

References 35 publications

Towards Identity Preserving Normal to Dysarthric Voice Conversion

Towards Identity Preserving Normal to Dysarthric Voice Conversion

Voicy: Zero-Shot Non-Parallel Voice Conversion in Noisy Reverberant Environments

Whispered and Lombard Neural Speech Synthesis

Contact Info

Product

Resources

About