Bidirectional LSTM Networks for Context-Sensitive Keyword Detection in a Cognitive Virtual Agent Framework

Abstract

Robustly detecting keywords in human speech is an important precondition for cognitive systems, which aim at intelligently interacting with users. Conventional techniques for keyword spotting usually show good performance when evaluated on well articulated read speech. However, modeling natural, spontaneous, and emotionally colored speech is challenging for today's speech recognition systems and thus requires novel approaches with enhanced robustness. In this article, we propose a new architecture for vocabulary independent keyword detection as needed for cognitive virtual agents such as the SEMAINE system. Our word spotting model is composed of a Dynamic Bayesian Network (DBN) and a bidirectional Long Short-Term Memory (BLSTM) recurrent neural net. The BLSTM network uses a self-learned amount of contextual information to provide a discrete phoneme prediction feature for the DBN, which is able to distinguish between keywords and arbitrary speech. We evaluate our Tandem BLSTM-DBN technique on both read speech and spontaneous emotional speech and show that our method significantly outperforms conventional Hidden Markov Model-based approaches for both application scenarios.

Automated Preamble Detection in Dictated Medical Reports

“…Many linguistic problems feature dependencies at longer distances, which LSTMs are better able to capture than convolutional or plain recurrent approaches. Bidirection LSTM (Bi-LSTM) networks (Graves and Schmidhuber, 2005;Graves et al, 2005;Wöllmer et al, 2010) also use future context, and recent work has shown advantages of Bi-LSTM networks for sequence labeling and named entity recognition (Huang et al, 2015;Chiu and Nichols, 2015;Wang et al, 2015;Lample et al, 2016;Ma and Hovy, 2016;Plank et al, 2016).…”

Online Driver Distraction Detection Using Long Short-Term Memory

“…Long Short-Term Memory networks have shown excellent performance in many pattern recognition disciplines [30]- [33].…”

Distributed Reservoir Computing with Sparse Readouts [Research Frontier]

“…Tasks that would benefit from such algorithms abound, including distributed multimedia classification [12], event detection with array of microphones [13], classification of texts in cluster environments [14] and prediction of highly nonlinear time-series in wireless sensor networks [4]. Still, it is known that training an RNN model is a challenging task even in a centralized context, which is far from being fully solved [15]- [18]. As such, there is a lack of available distributed protocols for RNN models satisfying all the requirements discussed above.…”