The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods

Lorenzo-Trueba, Jaime; Yamagishi, Junichi; Toda, Tomoki; Saito, Daisuke; Villavicencio, Fernando; Kinnunen, Tomi; Ling, Zhen-Hua

doi:10.48550/arxiv.1804.04262

Cited by 42 publications

(49 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To run this method, we used the source code provided by its author 2 . Note that this system was used as a baseline system in the Voice Conversion Challenge (VCC) 2018 [60].…”

Section: B Methods For Comparisonmentioning

confidence: 99%

FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversion

Kameoka,

Tanaka,

Kaneko

2021

Preprint

View full text Add to dashboard Cite

This paper proposes a non-autoregressive extension of our previously proposed sequence-to-sequence (S2S) modelbased voice conversion (VC) methods. S2S model-based VC methods have attracted particular attention in recent years for their flexibility in converting not only the voice identity but also the pitch contour and local duration of input speech, thanks to the ability of the encoder-decoder architecture with the attention mechanism. However, one of the obstacles to making these methods work in real-time is the autoregressive (AR) structure.To overcome this obstacle, we develop a method to obtain a model that is free from an AR structure and behaves similarly to the original S2S models, based on a teacher-student learning framework. In our method, called "FastS2S-VC", the student model consists of encoder, decoder, and attention predictor. The attention predictor learns to predict attention distributions solely from source speech along with a target class index with the guidance of those predicted by the teacher model from both source and target speech. Specifically, it is designed as a network that takes the mel-spectrogram of source speech as input and generates in parallel each row of an attention weight matrix represented by a Gaussian function. Thanks to the model structure that does not take the mel-spectrogram of target speech as input, the model is freed from an AR structure and allows for parallelization. Furthermore, we show that FastS2S-VC is suitable for real-time implementation based on a slidingwindow approach, and describe how to make it run in real-time. Through speaker-identity and emotional-expression conversion experiments, we confirmed that FastS2S-VC was able to speed up the conversion process by 70 to 100 times compared to the original AR-type S2S-VC methods, without significantly degrading the audio quality and similarity to target speech. We also confirmed that the real-time version of FastS2S-VC can be run with a latency of 32 ms when run on a GPU.

show abstract

“…To run this method, we used the source code provided by its author 2 . Note that this system was used as a baseline system in the Voice Conversion Challenge (VCC) 2018 [60].…”

Section: B Methods For Comparisonmentioning

confidence: 99%

FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversion

Kameoka,

Tanaka,

Kaneko

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Traditional voice conversion is studied for speech with a neutral expression, where voice quality has been the main focus. The most widely used speech databases for voice conversion include VCTK database [35], CMU-Arctic database [36], and Voice Conversion Challenge (VCC) corpus [37][38][39]. Since these speech databases are emotion-free, it is straightforward to pay attention to the vocal timbre when conducting voice conversion research.…”

Section: Traditional Voice Conversion and Datasetsmentioning

confidence: 99%

Expressive Voice Conversion: A Joint Framework for Speaker Identity and Emotional Style Transfer

Du¹,

Şişman²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Traditional voice conversion (VC) has been focused on speaker identity conversion for speech with a neutral expression. We note that emotional expression plays an essential role in daily communication, and the emotional style of speech can be speaker-dependent. In this paper, we study the technique to jointly convert the speaker identity and speakerdependent emotional style, that is called expressive voice conversion. We propose a StarGAN-based framework to learn a many-to-many mapping across different speakers, that takes into account speaker-dependent emotional style without the need for parallel data. To achieve this, we condition the generator on emotional style encoding derived from a pre-trained speech emotion recognition (SER) model. The experiments validate the effectiveness of our proposed framework in both objective and subjective evaluations. To our best knowledge, this is the first study on expressive voice conversion.

show abstract

“…We believe that the reason for this is the limitation to only use ASVspoof 2019 data. As presented in the ASVspoof 2021 workshop, the DF evaluation dataset contains data from the VCC 2018 [30] and VCC 2020 [16] contests. In order to maintain good results and updated systems, new datasets containing a large variety of distributions must be used in the training process of the models.…”

Section: Out Of Domain Data Df Partmentioning

confidence: 99%

A Study On Data Augmentation In Voice Anti-Spoofing

Cohen¹,

Rimon²,

Permuter³

2021

Preprint

View full text Add to dashboard Cite

In this paper we perform an in depth study of how data augmentation techniques improve synthetic or spoofed audio detection. Specifically, we propose methods to deal with channel variability, different audio compressions, different bandwidths and unseen spoofing attacks, which have all been shown to significantly degrade the performance of audio based systems and Anti-Spoofing systems. Our results are based on the ASVspoof 2021 challenge, in the Logical Access (LA) and Deep Fake (DF) categories. Our study is Data-Centric, meaning that the models are fixed and we significantly improve the results by making changes in the data. We introduce two forms of data augmentation -compression augmentation for the DF part, compression & channel augmentation for the LA part. In addition, a new type of online data augmentation, SpecAverage, is introduced in which the audio features are masked with their average value in order to improve generalization. Furthermore, we introduce a Log spectrogram feature design that improved the results. Our best single system and fusion scheme both achieve state of the art performance in the DF category, with an EER of 15.46% and 14.46% respectively. Our best system for the LA task reduced the best baseline EER by 50% and the min t-DCF by 16%. Our techniques to deal with spoofed data from a wide variety of distributions can be replicated and can help anti-spoofing and speech based systems enhance their results.

show abstract

The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods

Cited by 42 publications

References 3 publications

FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversion

FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversion

Expressive Voice Conversion: A Joint Framework for Speaker Identity and Emotional Style Transfer

A Study On Data Augmentation In Voice Anti-Spoofing

Contact Info

Product

Resources

About