Cross-Lingual Phoneme Mapping for Language Robust Contextual Speech Recognition

Patel, Ami; Li, David; Cho, Eunjoon; Aleksic, Petar S.

doi:10.1109/icassp.2018.8461600

Cited by 4 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to on-the-fly rescoring, we are able to dynamically modify the joint pronounciation/language model graph by adding new paths at runtime. This allows us to insert dynamic pronunciations or language model classes [3,4]. We do not yet have a direct replacement for this additional functionality in LAS.…”

Section: On-the-fly Rescoringmentioning

confidence: 99%

“…Conventional contextual systems rely on being able to inspect and modify individual components of modular systems in order to function. For example, a standalone language model can support dynamic population of classes [3], and a standalone pronunciation model allows dynamic injection of pronunciations [4]. One drawback of this is that information consumed within one piece of the modeling may be useful elsewhere; acoustic signals could inform a language model or text normalizer.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Contextual Speech Recognition in End-to-end Neural Network Systems Using Beam Search

Williams¹,

Kannan²,

Aleksic³

et al. 2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

Recent work has shown that end-to-end (E2E) speech recognition architectures such as Listen Attend and Spell (LAS) can achieve state-of-the-art quality results in LVCSR tasks. One benefit of this architecture is that it does not require a separately trained pronunciation model, language model, and acoustic model. However, this property also introduces a drawback: it is not possible to adjust language model contributions separately from the system as a whole. As a result, inclusion of dynamic, contextual information (such as nearby restaurants or upcoming events) into recognition requires a different approach from what has been applied in conventional systems. We introduce a technique to adapt the inference process to take advantage of contextual signals by adjusting the output likelihoods of the neural network at each step in the beam search. We apply the proposed method to a LAS E2E model and show its effectiveness in experiments on a voice search task with both artificial and real contextual information. Given optimal context, our system reduces WER from 9.2% to 3.8%. The results show that this technique is effective at incorporating context into the prediction of an E2E system.

show abstract

Section: On-the-fly Rescoringmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Contextual Speech Recognition in End-to-end Neural Network Systems Using Beam Search

Williams¹,

Kannan²,

Aleksic³

et al. 2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Cross-lingual phoneme mapping has been used in conventional systems for recognizing foreign words [15]. First, a phoneme mapping is learned by aligning the pronunciations between foreign and target languages using TTS-synthesized audio and a pronunciation learning algorithm [18].…”

Section: Phoneme Mappingmentioning

confidence: 99%

“…We train our model using only American English data and thus its wordpieces and phoneme set (no data from foreign languages). In inference, given a list of foreign words, we bias the recognition using an English phoneme-level biasing FST, which is built by first tokenizing the words into foreign phonemes and then mapping them to English phonemes using [15]. For example, given a navigation query "directions to Créteil" and the assumption that the French word "Créteil" is in our biasing list, "Créteil" is first tokenized to French phonemes as "k R e t E j", and then mapped to English phonemes "k r\ E t E j" for biasing 1 .…”

Section: Introductionmentioning

confidence: 99%

“…Although biasing is improved by these techniques, they do not address cross-lingual recognition. In [15], contextual biasing has been used to assist recognition of foreign words. With the phoneme mapping from a foreign language phoneme set to the recognizer's phoneme set, foreign words are modeled as a phoneme-level contextual FST for biasing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models

Jean

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

Contextual automatic speech recognition, i.e., biasing recognition towards a given context (e.g. user's playlists, or contacts), is challenging in end-to-end (E2E) models. Such models maintain a limited number of candidates during beam-search decoding, and have been found to recognize rare named entities poorly. The problem is exacerbated when biasing towards proper nouns in foreign languages, e.g., geographic location names, which are virtually unseen in training and are thus outof-vocabulary (OOV). While grapheme or wordpiece E2E models might have a difficult time spelling OOV words, phonemes are more acoustically salient and past work has shown that E2E phoneme models can better predict such words. In this work, we propose an E2E model containing both English wordpieces and phonemes in the modeling space, and perform contextual biasing of foreign words at the phoneme level by mapping pronunciations of foreign words into similar English phonemes. In experimental evaluations, we find that the proposed approach performs 16% better than a grapheme-only biasing model, and 8% better than a wordpiece-only biasing model on a foreign place name recognition task, with only slight degradation on regular English tasks.

show abstract

Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models

Bruguier

Sainath

et al. 2019

Preprint

View full text Add to dashboard Cite

Cross-Lingual Phoneme Mapping for Language Robust Contextual Speech Recognition

Cited by 4 publications

References 0 publications

Contextual Speech Recognition in End-to-end Neural Network Systems Using Beam Search

Contextual Speech Recognition in End-to-end Neural Network Systems Using Beam Search

Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models

Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models

Contact Info

Product

Resources

About