Deep LSTM based Feature Mapping for Query Classification

Abstract

Traditional convolutional neural network (CNN) based query classification uses linear feature mapping in its convolution operation. The recurrent neural network (RNN), differs from a CNN in representing word sequence with their ordering information kept explicitly. We propose using a deep long-short-term-memory (DLSTM) based feature mapping to learn feature representation for CNN. The DLSTM, which is a stack of LSTM units, has different order of feature representations at different depth of LSTM unit. The bottom LSTM unit equipped with input and output gates, extracts the first order feature representation from current word. To extract higher order nonlinear feature representation, the LSTM unit at higher position gets input from two parts. First part is the lower LSTM unit's memory cell from previous word. Second part is the lower LSTM unit's hidden output from current word. In this way, the DLSTM captures the nonlinear nonconsecutive interaction within n-grams. Using an architecture that combines a stack of the DLSTM layers with a tradition CNN layer, we have observed new state-of-the-art query classification accuracy on benchmark data sets for query classification.

Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures

“…User intent detectors classify user utterances to into one of the pre-defined intents. SVM, CNN and RNN models (Silva et al, 2011;Hashemi et al, 2016;Shi et al, 2016) perform well for intent classification. Belief trackers, which keep track of user goals and constraints every turn (Henderson et al, 2014a,b;Kim et al, 2017) are the most important component for task accomplishment.…”

A Linguistic Analysis of Visually Grounded Dialogues Based on Spatial Expressions

“…From theoretical aspects, various dialogue structures have been studied, including discourse structure (Stent, 2000;Asher et al, 2003), speech act (Austin, 1962;Searle, 1969) and common grounding (Clark, 1996;Lascarides and Asher, 2009). In dialogue system engineering, various linguistic structures have been considered and applied, including syntactic dependency (Davidson et al, 2019), predicate-argument structure (PAS) (Yoshino et al, 2011), ellipsis (Quan et al, 2019;Hansen and Søgaard, 2020), intent recognition (Silva et al, 2011;Shi et al, 2016), semantic representation/parsing (Mesnil et al, 2013;Gupta et al, 2018) and frame-based dialogue state tracking (Williams et al, 2016;El Asri et al, 2017). However, most prior work focus on dialogues where information is not grounded in external, perceptual modality such as vision.…”

POI Atmosphere Categorization Using Web Search Session Behavior

“…Note that we used the term "categorization" in this paper since our purpose is to group similar POIs either explicitly or implicitly, rather than just classify them. There have been many studies on query classification [4,5,9,14,16,21], since understanding the intents of users' queries is essential to improving the quality of a search engine from a practical standpoint. However, there are no studies connecting embeddings based on search query logs to geospatial features in the real world, like ours.…”

“…We explain previous studies on query classification in more detail, which are roughly divided into three groups [26] with respect to training features: query expression, retrieved content, and user behavior. The first group based on query expression uses common text features, such as the types, frequencies, and lengths of words and characters in a query [5,16]. An advantage of this information is that we can develop a classification system at a low cost by only considering query texts.…”