Generation of Large-Scale Simulated Utterances in Virtual Rooms to Train Deep-Neural Networks for Far-Field Speech Recognition in Google Home

Kim, Chanwoo; Misra, Ananya; Chin, K. K.; Hughes, T. A.; Narayanan, Arun; Sainath, Tara N.; Bacchiani, Michiel

doi:10.21437/interspeech.2017-1510

Cited by 217 publications

(187 citation statements)

References 0 publications

Supporting

Mentioning

184

Contrasting

Unclassified

Order By: Relevance

“…In addition to the diverse training sets described in Sec. 3.1 and 3.2, multi-condition training (MTR) [27,28] and random data downsampling to 8kHz [29] are also used to further increase data diversity. Noisy data is generated at signal-noise-ratio (SNR) from 0 to 30 dB, with an average SNR of 12 dB, and with T60 times ranging from 0 to 900 msec, averaging 500 msec.…”

Section: Methodsmentioning

confidence: 99%

A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency

Sainath

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

179

127

View full text Add to dashboard Cite

Thus far, end-to-end (E2E) models have not been shown to outperform state-of-the-art conventional models with respect to both quality, i.e., word error rate (WER), and latency, i.e., the time the hypothesis is finalized after the user stops speaking. In this paper, we develop a first-pass Recurrent Neural Network Transducer (RNN-T) model and a second-pass Listen, Attend, Spell (LAS) rescorer that surpasses a conventional model in both quality and latency. On the quality side, we incorporate a large number of utterances across varied domains [1] to increase acoustic diversity and the vocabulary seen by the model. We also train with accented English speech to make the model more robust to different pronunciations. In addition, given the increased amount of training data, we explore a varied learning rate schedule. On the latency front, we explore using the end-of-sentence decision emitted by the RNN-T model to close the microphone, and also introduce various optimizations to improve the speed of LAS rescoring. Overall, we find that RNN-T+LAS offers a better WER and latency tradeoff compared to a conventional model. For example, for the same latency, RNN-T+LAS obtains a 8% relative improvement in WER, while being more than 400-times smaller in model size.

show abstract

Section: Methodsmentioning

confidence: 99%

A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency

Sainath

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

179

127

View full text Add to dashboard Cite

show abstract

“…For a certain filterbank energy value e[m, c], let us define the following term η and the function f that is the ratio of e[m, c] to epeak in decibels (dB): Fig. 1a shows the probability density function of the timefrequency bins with respect to η defined in (3). In obtaining this distribution, we used a randomly chosen 1,000 utterances from the LibriSpeech training corpus [14].…”

Section: Distribution Of Energy In Time-frequency Bins Of Speech Signalsmentioning

confidence: 99%

“…These improvements have been obtained by the shift from Gaussian Mixture Model (GMM) to the Feed-Forward Deep Neural Networks (FF-DNNs), FF-DNNs to Recurrent Neural Network (RNN) such as the Long Short-Term Memory (LSTM) networks [2]. Thanks to these advances, voice assistant devices such as Google Home [3] , Amazon Alexa and Samsung Bixby [4] are widely used at home environments.…”

Section: Introductionmentioning

confidence: 99%

Small Energy Masking for Improved Neural Network Training for End-To-End Speech Recognition

Kim

Indurthi

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

In this paper, we present a Small Energy Masking (SEM) algorithm, which masks inputs having values below a certain threshold. More specifically, a time-frequency bin is masked if the filterbank energy in this bin is less than a certain energy threshold. A uniform distribution is employed to randomly generate the ratio of this energy threshold to the peak filterbank energy of each utterance in decibels. The unmasked feature elements are scaled so that the total sum of the feature values remain the same through this masking procedure. This very simple algorithm shows relatively 11.2 % and 13.5 % Word Error Rate (WER) improvements on the standard Lib-riSpeech test-clean and test-other sets over the baseline end-to-end speech recognition system. Additionally, compared to the input dropout algorithm, SEM algorithm shows relatively 7.7 % and 11.6 % improvements on the same LibriSpeech test-clean and test-other sets. With a modified shallow-fusion technique with a Transformer LM, we obtained a 2.62 % WER on the Lib-riSpeech test-clean set and a 7.87 % WER on the LibriSpeech test-other set.

show abstract

“…This improvement has come about from the shift from Gaussian Mixture Model (GMM) to the Feed-Forward Deep Neural Networks (FF-DNNs), FF-DNNs to Recurrent Neural Network (RNN) and in particular the Long Short-Term Memory (LSTM) networks [9]. Thanks to these advances, voice assistant devices such as Google Home [2,10] , Amazon Alexa or Samsung Bixby [11] are being used at many homes and on personal devices.…”

Section: Introductionmentioning

confidence: 99%

End-to-End Training of a Large Vocabulary End-to-End Speech Recognition System

Kim

Shin

Singh

et al. 2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

Self Cite

View full text Add to dashboard Cite

In this paper, we present an end-to-end training framework for building state-of-the-art end-to-end speech recognition systems.Our training system utilizes a cluster of Central Processing Units (CPUs) and Graphics Processing Units (GPUs). The entire data reading, large scale data augmentation, neural network parameter updates are all performed "on-the-fly". We use vocal tract length perturbation [1] and an acoustic simulator [2] for data augmentation. The processed features and labels are sent to the GPU cluster. The Horovod allreduce approach is employed to train neural network parameters. We evaluated the effectiveness of our system on the standard Librispeech corpus [3] and the 10,000-hr anonymized Bixby English dataset. Our end-to-end speech recognition system built using this training infrastructure showed a 2.44 % WER on test-clean of the LibriSpeech test set after applying shallow fusion with a Transformer language model (LM). For the proprietary English Bixby open domain test set, we obtained a WER of 7.92 % using a Bidirectional Full Attention (BFA) end-to-end model after applying shallow fusion with an RNN-LM. When the monotonic chunckwise attention (MoCha) based approach is employed for streaming speech recognition, we obtained a WER of 9.95 % on the same Bixby open domain test set.

show abstract

Generation of Large-Scale Simulated Utterances in Virtual Rooms to Train Deep-Neural Networks for Far-Field Speech Recognition in Google Home

Cited by 217 publications

References 0 publications

A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency

A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency

Small Energy Masking for Improved Neural Network Training for End-To-End Speech Recognition

End-to-End Training of a Large Vocabulary End-to-End Speech Recognition System

Contact Info

Product

Resources

About