Towards Identity Preserving Normal to Dysarthric Voice Conversion

Huang, Wen-Chin; Halpern, Bence Mark; Violeta, Lester Phillip; Scharenborg, Odette; Toda, Tomoki

doi:10.1109/icassp43922.2022.9747550

Cited by 12 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The TTS model simply fine-tuned a pretrained model on ELSIMU2. For the VC setup, we followed the same parallel VC using many-to-one setup as described in [47]. Since our dataset only contains one source healthy speaker, we generated multiple source healthy speakers using a pretrained TTS model to create a larger parallel corpus.…”

Section: B Implementation Detailsmentioning

confidence: 99%

Pretraining and Adaptation Techniques for Electrolaryngeal Speech Recognition

Violeta,

Ma,

Huang

et al. 2024

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

We investigate state-of-the-art automatic speech recognition (ASR) systems and provide thorough investigations on training methods to adapt them to low-resourced electrolaryngeal (EL) datasets. Transfer learning is often sufficient to resolve low-resourced problems; however, in EL speech, the domain shift between the pretraining and fine-tuning data is too large to overcome, limiting the ASR performance. We propose a method of reducing the domain shift gap during transfer learning between the healthy and EL datasets by conducting an intermediate fine-tuning task that uses imperfectly synthesized EL speech. Although using imperfect synthetic speech is nonintuitive, we proved the effectiveness of this method by decreasing the character error rate by up to 6.1% compared to the baselines using naive transfer learning. To further understand the model's behavior, we further analyze the produced latent spaces in each task through linguistic and identity proxy tasks and find that the intermediate fine-tuning focuses on identifying the voicing characteristics of the EL speakers. Moreover, we also show the differences between a simulated EL speaker from a real EL speaker and find that simulated EL data has pronunciation differences from real EL data, showing the huge domain gap between real EL and other speech data.

show abstract

Section: B Implementation Detailsmentioning

confidence: 99%

Pretraining and Adaptation Techniques for Electrolaryngeal Speech Recognition

Violeta,

Ma,

Huang

et al. 2024

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The large systematic mismatch between dysarthric and typical speech, high intraand inter-speaker variabilities, and data scarcity are three major challenges in ADSR. To enhance the performance of ADSR systems, previous studies have employed data augmentation, (e.g., speed perturbation [6,7] and voice conversion [8]), highlevel speech representations (e.g., bottleneck [9] and autoencoder bottleneck [10] features) and multi-modal representations † Equal contribution. Supported by EPSRC Project EP/R012180/1 (SpeechWave).…”

Section: Introductionmentioning

confidence: 99%

Dysarthric Speech Recognition, Detection and Classification using Raw Phase and Magnitude Spectra

Yue¹,

Loweimi²,

Cvetković³

2023

Interspeech 2023

View full text Add to dashboard Cite

In this paper, we explore the effectiveness of deploying the raw phase and magnitude spectra for dysarthric speech recognition, detection and classification. In particular, we scrutinise the usefulness of various raw phase-based representations along with their combinations with the raw magnitude spectrum and filterbank features. We employed single and multi-stream architectures consisting of a cascade of convolutional, recurrent and fully-connected layers for acoustic modelling. Furthermore, we investigate various configurations and fusion schemes as well as their training dynamics. In addition, the accuracies of the raw phase and magnitude based systems in the detection and classification tasks are studied and discussed. We report the performance on the UASpeech and TORGO dysarthric speech databases and for different severity levels. Our best system achieved WERs of 31.2% and 9.1% for dysarthric and typical speech on TORGO and 30.2% on UASpeech, respectively.

show abstract