Investigation into Target Speaking Rate Adaptation for Voice Conversion

Abstract

Disentangling speaker and content attributes of a speech signal into separate latent representations followed by decoding the content with an exchanged speaker representation is a popular approach for voice conversion, which can be trained with non-parallel and unlabeled speech data. However, previous approaches perform disentanglement only implicitly via some sort of information bottleneck or normalization, where it is usually hard to find a good trade-off between voice conversion and content reconstruction. Further, previous works usually do not consider an adaptation of the speaking rate to the target speaker or they put some major restrictions to the data or use case. Therefore, the contribution of this work is two-fold. First, we employ an explicit and fully unsupervised disentanglement approach, which has previously only been used for representation learning, and show that it allows to obtain both superior voice conversion and content reconstruction. Second, we investigate simple and generic approaches to linearly adapt the length of a speech signal, and hence the speaking rate, to a target speaker and show that the proposed adaptation allows to increase the speaking rate similarity with respect to the target speaker.

A Comparison of Discrete and Soft Speech Units for Improved Voice Conversion

“…The last experiment assesses naturalness, intelligibility and speaker similarity. We compare Urhythmic to three state-of-the-art unsupervised rhythm conversion systems: AutoPST [9], UnsupSeg [10], and DISSC [11]. We use the official pretrained models for each baseline.…”

“…However, training these systems requires parallel speech or text transcriptions, which are costly and time-consuming to collect. Unsupervised methods such as AutoPST [9], UnsupSeg [10], and DISSC [11] lift this restriction by modeling rhythm without annotations or parallel data. However, there is still a gap in quality and prosody compared to natural speech.…”

Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Units

“…Other methods only linearly alter the speaking rate (Kuhlmann et al, 2022) thus ignoring the change of rhythm for different content.…”

“…Traditional VC methods mainly focused on changing the timbre of a given speaker while leaving the speaking style unchanged (Stylianou et al, 1998;Kain and Macon, 1998;Nakashika et al, 2013;Chou et al, 2019;Huang et al, 2021). Recent methods propose to additionally convert speaking style (Qian et al, 2020;Chen and Duan, 2022;Qian et al, 2021;Kuhlmann et al, 2022). However, these mainly use only a single target utterance which does not fully capture speaker prosody.…”