The Academia Sinica Systems of Voice Conversion for VCC2020

Peng, Yuhuai; Cheng-Hung, Hu,; Kang, Alexander Chao-Fu; Hung-Shin, Lee,; Chen, Pin-Yuan; Tsao, Yu; Wang, Hsin‐Min

doi:10.21437/vcc_bc.2020-28

Cited by 4 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The typical usage of S3R models is to extract continuous [32]- [36], [42] features for downstream tasks. However, due to the lack of supervision, continuous S3Rs lack the ability to fully separate contents from other factors such as speaker identity, resulting in poor performance in the A2A setting [29].…”

Section: A Recognition-synthesis Based Voice Conversionmentioning

confidence: 99%

“…One way to provide the sufficient disentanglement is through discretization, as shown in [40]. Certain S3R models such as VQVAE [17] or vq-wav2vec [41] are able to generate discrete outputs due to their architecture, thus some have therefore proposed VC systems based on them [33], [42]. However, not all S3R models have such discretization design.…”

Section: A Recognition-synthesis Based Voice Conversionmentioning

confidence: 99%

See 1 more Smart Citation

A Comparative Study of Self-Supervised Speech Representation Based Voice Conversion

Huang

Yang

Hayashi

et al. 2022

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

We present a large-scale comparative study of selfsupervised speech representation (S3R)-based voice conversion (VC). In the context of recognition-synthesis VC, S3Rs are attractive owing to their potential to replace expensive supervised representations such as phonetic posteriorgrams (PPGs), which are commonly adopted by state-of-the-art VC systems. Using S3PRL-VC, an open-source VC software we previously developed, we provide a series of in-depth objective and subjective analyses under three VC settings: intra-/cross-lingual any-to-one (A2O) and any-to-any (A2A) VC, using the voice conversion challenge 2020 (VCC2020) dataset. We investigated S3R-based VC in various aspects, including model type, multilinguality, and supervision. We also studied the effect of a post-discretization process with k-means clustering and showed how it improves in the A2A setting. Finally, the comparison with state-of-the-art VC systems demonstrates the competitiveness of S3R-based VC and also sheds light on the possible improving directions.

show abstract

Section: A Recognition-synthesis Based Voice Conversionmentioning

confidence: 99%

Section: A Recognition-synthesis Based Voice Conversionmentioning

confidence: 99%

A Comparative Study of Self-Supervised Speech Representation Based Voice Conversion

Huang

Yang

Hayashi

et al. 2022

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

“…VQW2V: Features such as BNF are derived from an ASR model, which requires supervision using labels; this increases the cost of building such a system, especially in low-resource or crosslingual settings. Alternatively, several studies [17][18][19] adopted self-supervised representations that do not require any label during training while still being speaker-independent and framewise. In this work, we adopt the vector-quantized wav2vec (VQW2V) [20].…”

Section: Intermediate Representationmentioning

confidence: 99%

On Prosody Modeling for ASR+TTS based Voice Conversion

Huang¹,

Hayashi²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

In voice conversion (VC), an approach showing promising results in the latest voice conversion challenge (VCC) 2020 is to first use an automatic speech recognition (ASR) model to transcribe the source speech into the underlying linguistic contents; these are then used as input by a text-to-speech (TTS) system to generate the converted speech. Such a paradigm, referred to as ASR+TTS, overlooks the modeling of prosody, which plays an important role in speech naturalness and conversion similarity. Although some researchers have considered transferring prosodic clues from the source speech, there arises a speaker mismatch during training and conversion. To address this issue, in this work, we propose to directly predict prosody from the linguistic representation in a target-speaker-dependent manner, referred to as target text prediction (TTP). We evaluate both methods on the VCC2020 benchmark and consider different linguistic representations. The results demonstrate the effectiveness of TTP in both objective and subjective evaluations.

show abstract