LatticeRnn: Recurrent Neural Networks Over Lattices

Abstract

We present a new model called LATTICERNN, which generalizes recurrent neural networks (RNNs) to process weighted lattices as input, instead of sequences. A LATTICERNN can encode the complete structure of a lattice into a dense representation, which makes it suitable to a variety of problems, including rescoring, classifying, parsing, or translating lattices using deep neural networks (DNNs). In this paper, we use LATTICERNNs for a classification task: each lattice represents the output from an automatic speech recognition (ASR) component of a spoken language understanding (SLU) system, and we classify the intent of the spoken utterance based on the lattice embedding computed by a LATTICERNN. We show that making decisions based on the full ASR output lattice, as opposed to 1-best or n-best hypotheses, makes SLU systems more robust to ASR errors. Our experiments yield improvements of 13% over a baseline RNN system trained on transcriptions and 10% over an nbest list rescoring system for intent classification.

Joint Contextual Modeling for ASR Correction and Language Understanding

“…However, to the best of our knowledge they have not been used with Neural Semantic Parsers implemented by Recurrent Neural Networks (RNNs) or similar architectures. The closest work would be [8], who propose to traverse an input lattice in topological order and use the RNN hidden state of the lattice final state as the dense vector representing the entire lattice. However, word confusion networks provide a much better and more efficient solution thanks to token alignments.…”

Confidence Estimation for Black Box Automatic Speech Recognition Systems Using Lattice Recurrent Neural Networks

“…Those DAGs are highly flexible structures that can be additionally enriched with a wide range of features [21,22]. Recently there has been much interest in examining neural network extensions to DAGs and other general graph structures [23,18,24]. The key question that any such approach needs to answer is how information associated with multiple graph arcs or nodes is combined.…”

Voice Trigger Detection from Lvcsr Hypothesis Lattices Using Bidirectional Lattice Recurrent Neural Networks

“…"LatticeRNN" [7] was originally introduced for the task of classifying user intents in natural language processing. For a topologically-sorted hypothesis lattice, the feature vector of each arc is input to the neural network along with the state vector of the arc's source node, and the output of the neural network becomes the arc's state.…”

“…To actively overcome some of the LVCSR's errors, we consider the use of a statistical model that can interpret the hypothesis lattice in a discriminative, data-driven manner. We propose the use of a bidirectional version of "LatticeRNN" [7] for this purpose, and show that a significant gain in accuracy can be obtained compared to using the simple posterior probability.…”