Cross-Lingual Speaker Adaptation Using Domain Adaptation and Speaker Consistency Loss for Text-To-Speech Synthesis

Xin, Detai; Saito, Yuki; Takamichi, Shinnosuke; Koriyama, Tomoki; Saruwatari, Hiroshi

doi:10.21437/interspeech.2021-897

Cited by 7 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[12] narrows the gap by introducing multiple reference encoders that encode both the synthesized and ground-truth (GT) acoustic features and allowing gradients flow through the synthesized ones to cover unseen cases in training. In [13], a consistency loss over speaker identity is applied by computing distance of speaker embeddings of the synthesized and the GT mel-spectrogram, which helps to keep target speaker identity after the cross-lingual transfer.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving Cross-lingual Speech Synthesis with Triplet Training Scheme

Ye¹,

Zhou²,

Su³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent advances in cross-lingual text-to-speech (TTS) made it possible to synthesize speech in a language foreign to a monolingual speaker. However, there is still a large gap between the pronunciation of generated cross-lingual speech and that of native speakers in terms of naturalness and intelligibility. In this paper, a triplet training scheme is proposed to enhance the cross-lingual pronunciation by allowing previously unseen content and speaker combinations to be seen during training. Proposed method introduces an extra fine-tune stage with triplet loss during training, which efficiently draws the pronunciation of the synthesized foreign speech closer to those from the native anchor speaker, while preserving the non-native speaker's timbre. Experiments are conducted based on a state-of-the-art baseline cross-lingual TTS system and its enhanced variants. All the objective and subjective evaluations show the proposed method brings significant improvement in both intelligibility and naturalness of the synthesized cross-lingual speech.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Inspired by [13] and [12], we employ triplet loss [14] and propose a triplet training scheme to cover unseen cases. A triplet is composed of an anchor, a positive and a negative sample.…”

Section: Introductionmentioning

confidence: 99%

Improving Cross-lingual Speech Synthesis with Triplet Training Scheme

Ye¹,

Zhou²,

Su³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…[17] proposed to train a multi-lingual speech emotion recognition model with adversarial domain adaptation. [18] introduced a speaker-independent speech encoder with DAT to further assist in text-to-speech synthesis. [19,20] proposed to apply DAT on speaker recognition models to address the domain mismatch problem.…”

Section: Introductionmentioning

confidence: 99%

Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptation

Huang¹,

Yu-Kuan²,

Lee³

2022

Preprint

View full text Add to dashboard Cite

Speech distortions are a long-standing problem that degrades the performance of supervisely trained speech processing models. It is high time that we enhance the robustness of speech processing models to obtain good performance when encountering speech distortions while not hurting the original performance on clean speech. In this work, we propose to improve the robustness of speech processing models by domain adversarial training (DAT). We conducted experiments based on the SUPERB framework on five different speech processing tasks. In case we do not always have knowledge of the distortion types for speech data, we analyzed the binary-domain and multi-domain settings, where the former treats all distorted speech as one domain, and the latter views different distortions as different domains. In contrast to supervised training methods, we obtained promising results in target domains where speech data is distorted with different distortions including new unseen distortions introduced during testing.

show abstract