A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding

Abstract

Intent detection and slot filling are two main tasks for building a spoken language understanding (SLU) system. The two tasks are closely tied and the slots often highly depend on the intent. In this paper, we propose a novel framework for SLU to better incorporate the intent information, which further guides the slot filling. In our framework, we adopt a joint model with Stack-Propagation which can directly use the intent information as input for slot filling, thus to capture the intent semantic knowledge. In addition, to further alleviate the error propagation, we perform the token-level intent detection for the Stack-Propagation framework. Experiments on two publicly datasets show that our model achieves the state-of-the-art performance and outperforms other previous methods by a large margin. Finally, we use the Bidirectional Encoder Representation from Transformer (BERT) model in our framework, which further boost our performance in SLU task.

AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling

“…In the self-attentive encoder, following Qin et al (2019), we use BiLSTM with the self-attention mechanism to leverage both advantages of temporal features within word orders and contextual information. Bidirectional LSTM A bidirectional LSTM (BiLSTM) (Hochreiter and Schmidhuber, 1997) consists of two LSTM layers.…”

“…One is the ATIS dataset (Hemphill et al, 1990) and the other is SNIPS dataset (Coucke et al, 2018), which are widely used as benchmark in SLU research. Both datasets follow the same format and partition as in Goo et al (2018) and Qin et al (2019).…”

“…Take the utterance "Please play happy birthday" for example, the intent detection can be seen as a classification task to classify the intent label (i.e., PlayMusic) while the slot filling can be treated as a sequence labeling task to predict the slot label sequence (i.e., O, O, B-music, I-music). Dominant SLU systems in the literature (Goo et al, 2018;E et al, 2019;Qin et al, 2019) adopt joint * Email corresponding. models to model the relation between the two tasks, which is a direction we follow.…”

“…Unlike the prior single intent SLU model which can simply leverage the utterance's single intent to guide slot prediction (Goo et al, 2018;Qin et al, 2019), multi-intent SLU faces to multiple intents and presents a unique challenge that is worth studying: how to effectively incorporate multiple intents information to lead the slot prediction. To this end, Gangadharaiah and Narayanaswamy (2019) first explored the multi-task framework with the slot-gated mechanism (Goo et al, 2018) for joint multiple intent detection and slot filling.…”

Winners at W-NUT 2020 Shared Task-3: Leveraging Event Specific and Chunk Span information for Extracting COVID Entities from Tweets

“…Taking inspiration from the same, we use CT-Bert as part of our architecture. A variety of slot-filling approaches have been built on top of these deep learning advancements (Kurata et al, 2016;Qin et al, 2019). The proposed baseline for our task (Zong et al, 2020) modifies Bert model for slot-filling problem inspired by Baldini Soares et al (2019).…”