Improving Performance of End-to-End ASR on Numeric Sequences

Abstract

Recognizing written domain numeric utterances (e.g., I need $1.25.) can be challenging for ASR systems, particularly when numeric sequences are not seen during training. This out-ofvocabulary (OOV) issue is addressed in conventional ASR systems by training part of the model on spoken domain utterances (e.g., I need one dollar and twenty five cents.), for which numeric sequences are composed of in-vocabulary numbers, and then using an FST verbalizer to denormalize the result. Unfortunately, conventional ASR models are not suitable for the low memory setting of on-device speech recognition. E2E models such as RNN-T are attractive for on-device ASR, as they fold the AM, PM and LM of a conventional model into one neural network. However, in the on-device setting the large memory footprint of an FST denormer makes spoken domain training more difficult. In this paper, we investigate techniques to improve E2E model performance on numeric data. We find that using a text-to-speech system to generate additional numeric training data, as well as using a small-footprint neural network to perform spoken-to-written domain denorming, yields improvement in several numeric classes. In the case of the longest numeric sequences, we see reduction of WER by up to a factor of 8.

Neural Inverse Text Normalization

“…Therefore, we employ FST-based text normalization methods to automatically normalize written form of text. This is similar to synthetic data generation employed successfully in the past [10]. However, the data prepared in such a way poses a number of problems for modeling ITN:…”

Recent Advances in End-to-End Automatic Speech Recognition

“…Because LM fusion methods require interpolating with an external LM, both the computational cost and footprint are increased, which is not applicable to ASR on devices. With the advance of TTS technologies, a new trend is to adapt E2E models with the synthesized speech generated from the new-domain text [12,156,177,178]. This is especially useful for adapting RNN-T, in which the prediction network works similarly to an LM.…”

Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future

“…For example, if a system that is intended for use in a voicemail transcription setting achieves 3% overall WER, but it mistranscribes every phone number, that system would almost certainly not be preferred over a system that achieves 3.5% overall WER, but that makes virtually no mistakes on phone numbers. As Peyser et al (2019) show, such examples are far from theoretical; fortunately, as they show, it is also possible to create synthetic test sets using text-to-speech systems to get a sense of WER in a specific context. Standard tools like NIST SCLITE 3 can be used to calculate WER and various additional statistics.…”