Lexicon-Free Conversational Speech Recognition with Neural Networks

Abstract

We present an approach to speech recognition that uses only a neural network to map acoustic input to characters, a character-level language model, and a beam search decoding procedure. This approach eliminates much of the complex infrastructure of modern speech recognition systems, making it possible to directly train a speech recognizer using errors generated by spoken language understanding tasks. The system naturally handles out of vocabulary words and spoken word fragments. We demonstrate our approach using the challenging Switchboard telephone conversation transcription task, achieving a word error rate competitive with existing baseline systems. To our knowledge, this is the first entirely neural-network-based system to achieve strong speech transcription results on a conversational speech task. We analyze qualitative differences between transcriptions produced by our lexicon-free approach and transcriptions produced by a standard speech recognition system. Finally, we evaluate the impact of large context neural network character language models as compared to standard n-gram models within our framework.

Advances in all-neural speech recognition

“…In contrast to this, we follow recent work [22,18,23,19] where a neural network learns context-dependence implicitly.…”

“…Our approach is most similar to the CTC methods of [20,28,19,18,17]. In contrast to [20,28,17], we use a ReLU-RNN rather than an LSTM, and find it to be effective and much faster.…”

“…Two main approaches have been used, both of which attempt to leverage a recurrent neural network's potential ability to "remember" information for a long period of time 1 and then act on it [12,13,14]. The first of these approaches uses an RNN trained in the "connectionist temporal classification" (CTC) framework [15] to predict letter rather than phonetic output [16,17,18,19,20]. At its base, this approach has no decoder at all, with all logic related to language modeling and decoding being implicitly done by the RNN.…”

Character-based units for unlimited vocabulary continuous speech recognition

“…There have been interesting developments in the creation of 'end-to-end' speech recognition systems [5,20] that create joint audio and language models, often working with the character as basic unit. In contrast to these systems, we explore the use of characters in a 'conventional' state-of-the-art LVCSR system.…”

Hybrid methodological approach to context-dependent speech recognition

“…domain-dependent) manner. [9][10][11][12][13] To reduce the limitations of the statistical approach, we employ a representational approach based on the focus tree model of attentional information in human-machine interaction. Various adaptations of this model were successfully applied in several prototypical conversational agents for the purposes of natural language understanding and dialogue management.…”