Disentangling Prosody Representations with Unsupervised Speech Reconstruction

Qu, Leyuan; Li, Taihao; Weber, Cornelius; Pekarek-Rosin, Theresa; Ren, Fuji; Wermter, Stefan

doi:10.48550/arxiv.2212.06972

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6 in Appendix A.5). In addition, two concurrent methods for SSC (Qu et al, 2022; were proposed recently. Although showing impressive results, they are based on spectrogram representations, hence require an additional vocoding step.…”

Section: Related Workmentioning

confidence: 99%

“…However, this limits the application to high resource languages and requires large scale data labelling. Some recent prosody aware VC methods use spectrogram representations as input and output (Qu et al, 2022;, rather than operating in the waveform domain. Thus, they involve another phase of converting from the spectral domain to the time domain using a vocoder.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Units

Maimon,

Adi

2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

We introduce DISSC, a novel, lightweight method that converts the rhythm, pitch contour and timbre of a recording to a target speaker in a textless manner. Unlike DISSC, most voice conversion (VC) methods focus primarily on timbre, and ignore people's unique speaking style (prosody). The proposed approach uses a pretrained, self-supervised model for encoding speech to discrete units, which makes it simple, effective, and fast to train. All conversion modules are only trained on reconstruction like tasks, thus suitable for any-to-many VC with no paired data. We introduce a suite of quantitative and qualitative evaluation metrics for this setup, and empirically demonstrate that DISSC significantly outperforms the evaluated baselines. Code and samples are available at https://pages.cs.huji.ac.il/adiyoss-lab/dissc/.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%