SynthASR: Unlocking Synthetic Data for Speech Recognition

Fazel, Amin; Yang, Wei; Liu, Yulan; Barra-Chicote, Roberto; Meng, Yixiong; Maas, Roland; Droppo, Jasha

doi:10.21437/interspeech.2021-1882

Cited by 23 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Synthetic speech generation techniques have recently gained attention in other related fields. Fazel et al [21] use synthetic speech generated with T2S to improve accuracy in ASR. Huang et al [62] use a machine translation technique to generate text to train an ASR language model in a low-resource language.…”

Section: Synthetic Speech Generation For Pronunciation Error Detectionmentioning

confidence: 99%

“…The probability of pronunciation errors for all the words in a sentence can then be calculated using the Bayes rule [18]. In this new formulation, we move the complexity to learning the speech generation process that is well suited to the problem of limited speech availability [19][20][21]. The proposed method outperforms the state-of-the-art model [9] in detecting pronunciation errors in AUC metric by 41% from 0.528 to 0.749 on the GUT Isle Corpus of L2 Polish speakers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Korzekwa,

Lorenzo-Trueba,

Drugman

et al. 2022

Preprint

View full text Add to dashboard Cite

The research community has long studied computer-assisted pronunciation training (CAPT) methods in non-native speech. Researchers focused on studying various model architectures, such as Bayesian networks and deep learning methods, as well as on the analysis of different representations of the speech signal. Despite significant progress in recent years, existing CAPT methods are not able to detect pronunciation errors with high accuracy (only 60% precision at 40%-80% recall). One of the key problems is the low availability of mispronounced speech that is needed for the reliable training of pronunciation error detection models. If we had a generative model that could mimic non-native speech and produce any amount of training data, then the task of detecting pronunciation errors would be much easier. We present three innovative techniques based on phoneme-to-phoneme (P2P), text-to-speech (T2S), and speech-to-speech (S2S) conversion to generate correctly pronounced and mispronounced synthetic speech. We show that these techniques not only improve the accuracy of three machine learning models for detecting pronunciation errors but also help establish a new state-of-the-art in the field. Earlier studies have used simple speech generation techniques such as P2P conversion, but only as an additional mechanism to improve the accuracy of pronunciation error detection. We, on the other hand, consider speech generation to be the first-class method of detecting pronunciation errors. The effectiveness of these techniques is assessed in the tasks of detecting pronunciation and lexical stress errors. Non-native English speech corpora of German, Italian, and Polish speakers are used in the evaluations. The best proposed S2S technique improves the accuracy of detecting pronunciation errors in AUC metric by 41% from 0.528 to 0.749 compared to the state-of-the-art approach.

show abstract

Section: Synthetic Speech Generation For Pronunciation Error Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Korzekwa,

Lorenzo-Trueba,

Drugman

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Data augmentation using either labelled or unlabelled data [1,2] has been used to alleviate this data scarcity problem. One promising approach is speech data synthesis, which recently contributed to significant progress in domain adaptation [3], medication names recognition [4], accurate numeric sequences transcription [5], low-resource languages [6], etc. Most research in this areas has focused on text-tospeech (TTS) generation [7,8,9,10] and audio augmentation [11,12,13], and their effects on the resulting ASR accuracy.…”

Section: Introductionmentioning

confidence: 99%

G-Augment: Searching for the Meta-Structure of Data Augmentation Policies for ASR

Wang

Cubuk

Rosenberg

et al. 2023

2022 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Aiming at reducing the reliance on expensive human annotations, data synthesis for Automatic Speech Recognition (ASR) has remained an active area of research. While prior work mainly focuses on synthetic speech generation for ASR data augmentation, its combination with text generation methods is considerably less explored. In this work, we explore text augmentation for ASR using large-scale pre-trained neural networks, and systematically compare those to traditional text augmentation methods. The generated synthetic texts are then converted to synthetic speech using a text-to-speech (TTS) system and added to the ASR training data. In experiments conducted on three datasets, we find that neural models achieve 9%-15% relative WER improvement and outperform traditional methods. We conclude that text augmentation, particularly through modern neural approaches, is a viable tool for improving the accuracy of ASR systems.

show abstract

“…Speech synthesis modelling techniques are advanced such that it is feasible to achieve almost natural-sounding output if sufficient data is available [1,2]. Specifically, current TTS techniques mostly require high-quality single-speaker recordings with text transcription for at least 2 hours of speech [3].…”

Section: Introductionmentioning

confidence: 99%

Building African Voices

Perez¹,

Neubig²,

Black³

2022

Preprint

View full text Add to dashboard Cite

Modern speech synthesis techniques can produce naturalsounding speech given sufficient high-quality data and compute resources. However, such data is not readily available for many languages. This paper focuses on speech synthesis for low-resourced African languages, from corpus creation to sharing and deploying the Text-to-Speech (TTS) systems. We first create a set of general-purpose instructions on building speech synthesis systems with minimum technological resources and subject-matter expertise. Next, we create new datasets and curate datasets from "found" data (existing recordings) through a participatory approach while considering accessibility, quality, and breadth. We demonstrate that we can develop synthesizers that generate intelligible speech with 25 minutes of created speech, even when recorded in suboptimal environments. Finally, we release the speech data, code, and trained voices for 12 African languages to support researchers and developers.

show abstract

SynthASR: Unlocking Synthetic Data for Speech Recognition

Cited by 23 publications

References 0 publications

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

G-Augment: Searching for the Meta-Structure of Data Augmentation Policies for ASR

Building African Voices

Contact Info

Product

Resources

About