Auxiliary Capsules for Natural Language Understanding

Staliūnaitė, Ieva; Iacobacci, Ignacio

doi:10.1109/icassp40776.2020.9053899

Cited by 6 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The method of using SLU as fine-tuning with pre-training on another task, or vice versa, has shown improvements in the SLU performance. However the results of [Staliūnaitė and Iacobacci 2020], echoing those of [Louvan and Magnini 2019] on slot tagging, indicate a parsimonious approach to adding extra tasks simultaneously more often yields a better result.…”

Section: Target Variationsmentioning

confidence: 93%

“…This explicit and direct feedback is stronger than the implicit or indirect joint learning typically found in RNN models. [Staliūnaitė and Iacobacci 2020] extended this work to a multi-task setting with extra mid-level capsules for NER and POS labels, with mixed results. [Wen et al 2018] Using hierarchy and context Two layer (Bi)LSTM [Wang et al 2018c] Capturing local semantic information CNN, BiLSTM encoder decoder [Firdaus et al 2018a] Domain dependence Ensemble model, GRU Slow training time Progressive multi-task model using user information [Li et al 2018a] Correlation of different tasks Multi-task model incl.…”

Section: Hierarchical Modelsmentioning

confidence: 99%

“…CRF and self attention over BERT [Ni et al 2020] Model transfer, IoT Pipeline structure from medical analogue [Krone et al 2020] Unseen labels Few-shot meta-learning [Bhathiya and Thayasivam 2020] Unseen labels, language transfer Few-shot meta-learning [Tang et al 2020] Linear chain CRF limitations GCN based CRF [Staliūnaitė and Iacobacci 2020] Extend capsule network Capsule network with MTL [Han et al 2020] Explicit interaction, word-label information Bidirection, attention, GCN graph 5.2.6 Bi-directional models. A model where there is a pipeline from one sub-task to the other may be seen as unidirectional.…”

Section: Hierarchical Modelsmentioning

confidence: 99%

“…[ Staliūnaitė and Iacobacci 2020] incorporated POS and NER tagging simultaneously with slot tagging and intent detection using a capsule network, however the results were generally poorer when both NER and POS were included rather than just one, and mixed for different data sets indicating a generalisability issue.…”

Section: Target Variationsmentioning

confidence: 99%

See 3 more Smart Citations

A survey of joint intent detection and slot-filling models in natural language understanding

Weld¹,

Huang²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Intent classification and slot filling are two critical tasks for natural language understanding. Traditionally the two tasks have been deemed to proceed independently. However, more recently, joint models for intent classification and slot filling have achieved state-of-the-art performance, and have proved that there exists a strong relationship between the two tasks. This article is a compilation of past work in natural language understanding, especially joint intent classification and slot filling. We observe three milestones in this research so far: Intent detection to identify the speaker's intention, slot filling to label each word token in the speech/text, and finally, joint intent classification and slot filling tasks. In this article, we describe trends, approaches, issues, data sets, evaluation metrics in intent classification and slot filling. We also discuss representative performance values, describe shared tasks, and provide pointers to future work, as given in prior works. To interpret the state-of-the-art trends, we provide multiple tables that describe and summarise past research along different dimensions, including the types of features, base approaches, and dataset domain used.

show abstract

Section: Target Variationsmentioning

confidence: 93%

Section: Hierarchical Modelsmentioning

confidence: 99%

Section: Hierarchical Modelsmentioning

confidence: 99%

Section: Target Variationsmentioning

confidence: 99%

See 2 more Smart Citations

A survey of joint intent detection and slot-filling models in natural language understanding

Weld¹,

Huang²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, many previous research articles have been published and have discussed the researcher's implementation on the topic of ID and IC with slot filling. Papers [1,8,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37] have discussed and presented their approach or implementation on developing ID and IC with slot filling using various frameworks, methodologies, and techniques. Various frameworks, methodologies, approaches, techniques and algorithms have been implemented by these researchers, whereas for instance, [19,22,24,26,27,28,31,34,35,36] have implemented techniques of Bidirectional Long Short-Term Memory ID (BiLSTM) with Conditional Random Forest (CRF) in their implementation of ID and IC with slot filling.…”

Section: Introductionmentioning

confidence: 99%

State of the Art in Intent Detection and Slot Filling for Question Answering System: A Systematic Literature Review

Zailan,

Teo,

Abdullah

et al. 2023

IJACSA

View full text Add to dashboard Cite

A Question Answering System (QAS), also known as a chatbot, is a Natural Language Processing (NLP) application that automatically provides accurate responses to questions posed by humans in natural language. Intent Detection and Classification are crucial elements in NLP, especially in a taskoriented dialogue system. In this paper, we conduct a systematic literature review that will perform a comparative analysis of different techniques or algorithms that are being implemented for intent detection and classification with slot filling. The goals of this paper are to identify the distribution, methodology, techniques or algorithms, and evaluation methods, that can be used to develop and construct a model of intent detection and classification with slot filling. This paper also reviews academic documents that have been published from 2019 to 2023, based on a four-step selection process of identification, screening, eligibility, and inclusion, for the selection process. In order to examine these documents, a systematic review was conducted and four main research questions were answered. The results discuss the methodology that can be used for the implementation of intent detection and classification with slot filling, along with the techniques, algorithms and evaluation methods that are widely used and currently implemented by other researchers.

show abstract

A Concurrent Intelligent Natural Language Understanding Model for an Automated Inquiry System

Sunilkumar¹,

Srihari²,

Gilbert³

et al. 2022

2022 IEEE World Conference on Applied Intelligence and Computing (AIC)

View full text Add to dashboard Cite

Auxiliary Capsules for Natural Language Understanding

Cited by 6 publications

References 16 publications

A survey of joint intent detection and slot-filling models in natural language understanding

A survey of joint intent detection and slot-filling models in natural language understanding

State of the Art in Intent Detection and Slot Filling for Question Answering System: A Systematic Literature Review

A Concurrent Intelligent Natural Language Understanding Model for an Automated Inquiry System

Contact Info

Product

Resources

About