Phoneme-to-Grapheme Conversion Based Large-Scale Pre-Training for End-to-End Automatic Speech Recognition

Masumura, Ryo; Makishima, Naoki; Ihori, Mana; Takashima, Akihiko; Tanaka, Tomohiro; Orihashi, Shota

doi:10.21437/interspeech.2020-1930

Cited by 8 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most straightforward approach is to simply use multi-speaker TTS to generate the waveform with various acoustic variations [314]- [318]. The other approaches are based on the generation of high-level (more linguistic) features instead of generating the waveform, e.g., encoder features [319] and phoneme features [320], [321]. This approach is similar to the back-translation technique developed in neural machine translation [322].…”

Section: From Representation Learning To Zero Resourcesmentioning

confidence: 99%

Self-Supervised Speech Representation Learning: A Review

Mohamed

Lee

Borgholt

et al. 2022

IEEE J. Sel. Top. Signal Process.

169

View full text Add to dashboard Cite

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. Users may download and print one copy of any publication from the public portal for the purpose of private study or research.  You may not further distribute the material or use it for any profit-making activity or commercial gain  You may freely distribute the URL identifying the publication in the public portal If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

show abstract

Section: From Representation Learning To Zero Resourcesmentioning

confidence: 99%

Self-Supervised Speech Representation Learning: A Review

Mohamed

Lee

Borgholt

et al. 2022

IEEE J. Sel. Top. Signal Process.

169

View full text Add to dashboard Cite

show abstract

“…The most straightforward approach is to simply use multi-speaker TTS to generate the waveform with various acoustic variations [293]- [297]. The other approaches are based on the generation of high-level (more linguistic) features instead of generating the waveform, e.g., encoder features [298] and phoneme features [299], [300]. This approach is similar to the back-translation technique developed in NMT [301].…”

Section: Robustness and Transferabilitymentioning

confidence: 99%

Self-Supervised Speech Representation Learning: A Review

Mohamed¹,

Lee²,

Borgholt³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…To generate acoustically similar errors on text data, we leverage phone information that can be automatically obtained from word sequences using a pronunciation dictionary [38,39]. We propose a modified version of ELECTRA called phone-attentive ELECTRA (P-ELECTRA).…”

Section: Phone-attentive Electramentioning

confidence: 99%

ASR Rescoring and Confidence Estimation with ELECTRA

Futami¹,

Inaguma²,

Mimura³

et al. 2021

Preprint

View full text Add to dashboard Cite

In automatic speech recognition (ASR) rescoring, the hypothesis with the fewest errors should be selected from the n-best list using a language model (LM). However, LMs are usually trained to maximize the likelihood of correct word sequences, not to detect ASR errors. We propose an ASR rescoring method for directly detecting errors with ELECTRA, which is originally a pre-training method for NLP tasks. ELECTRA is pre-trained to predict whether each word is replaced by BERT or not, which can simulate ASR error detection on large text corpora. To make this pre-training closer to ASR error detection, we further propose an extended version of ELECTRA called phone-attentive ELECTRA (P-ELECTRA). In the pre-training of P-ELECTRA, each word is replaced by a phone-to-word conversion model, which leverages phone information to generate acoustically similar words. Since our rescoring method is optimized for detecting errors, it can also be used for word-level confidence estimation. Experimental evaluations on the Librispeech and TED-LIUM2 corpora show that our rescoring method with ELECTRA is competitive with conventional rescoring methods with faster inference. ELECTRA also performs better in confidence estimation than BERT because it can learn to detect inappropriate words not only in fine-tuning but also in pre-training.

show abstract

Phoneme-to-Grapheme Conversion Based Large-Scale Pre-Training for End-to-End Automatic Speech Recognition

Cited by 8 publications

References 21 publications

Self-Supervised Speech Representation Learning: A Review

Self-Supervised Speech Representation Learning: A Review

Self-Supervised Speech Representation Learning: A Review

ASR Rescoring and Confidence Estimation with ELECTRA

Contact Info

Product

Resources

About