A Deep Learning Based Multi-task Ensemble Model for Intent Detection and Slot Filling in Spoken Language Understanding

Firdaus, Mauajama; Bhatnagar, Shobhit; Ekbal, Asif; Bhattacharyya, Pushpak

doi:10.1007/978-3-030-04212-7_57

Cited by 12 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A special token is added to encapsulate the whole utterance for use in intent classification [Hakkani-Tür et al 2016]; • a Bi-LSTM encoder decoder but with separate losses for intent and slot prediction ( [Zheng et al 2017]; • rather than seq2seq [Kim et al 2017] perform a global slot prediction (learning the joint distribution) from a matrix of the hidden states to a matrix of slot tag probabilities for each word, intent is predicted from a sum of hidden states; • [Wen et al 2018] propose to use both a hierarchical (multilayer) and a contextual (BiLSTM or LSTM) approach, investigating various combinations and using differing layers for intent and slot prediction; • an ensemble using both BiLSTM and BiGRU fed to separate MLPs whose outputs are fused then projected and a softmax applied to predict intent and slots concurrently is proposed by [Firdaus et al 2018a].…”

Section: Recurrent Neural Networkmentioning

confidence: 99%

“…[Staliūnaitė and Iacobacci 2020] extended this work to a multi-task setting with extra mid-level capsules for NER and POS labels, with mixed results. [Wen et al 2018] Using hierarchy and context Two layer (Bi)LSTM [Wang et al 2018c] Capturing local semantic information CNN, BiLSTM encoder decoder [Firdaus et al 2018a] Domain dependence Ensemble model, GRU Slow training time Progressive multi-task model using user information [Li et al 2018a] Correlation of different tasks Multi-task model incl. POS tag [Li et al 2018b] Sharing semantic information Self-attention Tagging strategy Token tags include intent and slot [Zhang et al 2019a] Hierarchical structure Capsule network with rerouting (feedback) Spatial (context) and serial (order) information Encoder-decoder, CNN [Wang et al 2018b] slot2intent and intent2slot Bi-directional architecture [Siddhant et al 2019] Unsupervised learning ELMo on unused utterances, BiLSTM Use sequence labelling output for intent Cross attention, BiLSTM, CRF Hierarchical vector approach Learn vectors representing elements of frame [Jung et al 2018] Model relationship between text and its semantic frame…”

Section: Hierarchical Modelsmentioning

confidence: 99%

“…The gamut of word embedding methods have been used including word2vec ( [Pan et al 2018;Wang et al 2018c]), fastText ([Firdaus et al 2020]), GloVe ([Bhasin et al 2019;Bhasin et al 2020;Dadas et al 2019;Liu et al 2019b;Okur et al 2019;Pentyala et al 2019;Thi Do and Gaspers 2019;Zhang and Wang 2016]), ELMo [Zhang et al 2020b] and [Krone et al 2020] (pre-print only), BERT ([Ni et al 2020;Qin et al 2019;] and Krone et al 2020] (pre-print only) and [Han et al 2020] (submitted for publication). [Firdaus et al 2018a] and [Firdaus et al 2019] used concatenated GloVe and word2vec embeddings to capture more word information.…”

Section: Token Embeddingmentioning

confidence: 99%

“…Pre-computed syntactic features, for example POS tags for each token using the nltk library ([Firdaus et al 2018a] have been included with word embeddings.…”

Section: Token Embeddingmentioning

confidence: 99%

See 3 more Smart Citations

A survey of joint intent detection and slot-filling models in natural language understanding

Weld¹,

Huang²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Intent classification and slot filling are two critical tasks for natural language understanding. Traditionally the two tasks have been deemed to proceed independently. However, more recently, joint models for intent classification and slot filling have achieved state-of-the-art performance, and have proved that there exists a strong relationship between the two tasks. This article is a compilation of past work in natural language understanding, especially joint intent classification and slot filling. We observe three milestones in this research so far: Intent detection to identify the speaker's intention, slot filling to label each word token in the speech/text, and finally, joint intent classification and slot filling tasks. In this article, we describe trends, approaches, issues, data sets, evaluation metrics in intent classification and slot filling. We also discuss representative performance values, describe shared tasks, and provide pointers to future work, as given in prior works. To interpret the state-of-the-art trends, we provide multiple tables that describe and summarise past research along different dimensions, including the types of features, base approaches, and dataset domain used.

show abstract

Section: Recurrent Neural Networkmentioning

confidence: 99%

Section: Hierarchical Modelsmentioning

confidence: 99%

Section: Token Embeddingmentioning

confidence: 99%

“…Pre-computed syntactic features, for example POS tags for each token using the nltk library ([Firdaus et al 2018a] have been included with word embeddings.…”

Section: Token Embeddingmentioning

confidence: 99%

See 2 more Smart Citations

A survey of joint intent detection and slot-filling models in natural language understanding

Weld¹,

Huang²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A classical example of slot filling with the sentence Show me the flights from Boston to New York today is shown in Table 1. While these tasks were previously treated separately, recent research have shown that joint models capable of answering to both tasks performed better [11,39,40].…”

Section: Intelligent Assistantsmentioning

confidence: 99%

User-in-the-loop adaptive intent detection for instructable digital assistant

Lair

Delgrange²,

Mugisha³

et al. 2020

Proceedings of the 25th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

People are becoming increasingly comfortable using Digital Assistants (DAs) to interact with services or connected objects. However, for non-programming users, the available possibilities for customizing their DA are limited and do not include the possibility of teaching the assistant new tasks. To make the most of the potential of DAs, users should be able to customize assistants by instructing them through Natural Language (NL). To provide such functionalities, NL interpretation in traditional assistants should be improved: (1) The intent identification system should be able to recognize new forms of known intents, and to acquire new intents as they are expressed by the user. ( 2) In order to be adaptive to novel intents, the Natural Language Understanding module should be sample efficient, and should not rely on a pretrained model. Rather, the system should continuously collect the training data as it learns new intents from the user. In this work, we propose AidMe (Adaptive Intent Detection in Multi-Domain Environments), a user-in-the-loop adaptive intent detection framework that allows the assistant to adapt to its user by learning his intents as their interaction progresses. AidMe builds its repertoire of intents and collects data to train a model of semantic similarity evaluation that can discriminate between the learned intents and autonomously discover new forms of known intents. AidMe addresses two major issues -intent learning and user adaptation -for instructable digital assistants. We demonstrate the capabilities of AidMe as a standalone system by comparing it with a one-shot learning system and a pretrained NLU module through simulations of interactions with a user. We also show how AidMe can smoothly integrate to an existing instructable digital assistant.

show abstract

RoboNLU: Advancing Command Understanding with a Novel Lightweight BERT-Based Approach for Service Robotics

Wang,

Neau,

Buche

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A Deep Learning Based Multi-task Ensemble Model for Intent Detection and Slot Filling in Spoken Language Understanding

Cited by 12 publications

References 26 publications

A survey of joint intent detection and slot-filling models in natural language understanding

A survey of joint intent detection and slot-filling models in natural language understanding

User-in-the-loop adaptive intent detection for instructable digital assistant

RoboNLU: Advancing Command Understanding with a Novel Lightweight BERT-Based Approach for Service Robotics

Contact Info

Product

Resources

About