An End-to-End Dialogue State Tracking System with Machine Reading Comprehension and Wide &amp; Deep Classification

Ma, Yue; Zeng, Zengfeng; Zhu, Dawei; Li, Xuan; Yang, Yiying; Yao, Xiaoyuan; Zhou, Kaijie; Shen, Jianping

doi:10.48550/arxiv.1912.09297

Cited by 19 publications

(14 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The deep neural networks have proved highly effective for many critical NLP tasks Farooq et al, 2020;Williams, 2019;Ma et al, 2019;Siddique et al, 2021;Liu and Lane, 2016;. We organize the related work on intent detection into three categories: (i) supervised intent detection, (ii) standard zero-shot intent detection, and (iii) generalized zero-shot intent detection.…”

Section: Related Workmentioning

confidence: 99%

“…In contrast, we leverage rich commonsense knowledge graph to capture deep semantic and discriminative relationships between utterances and intents, which significantly reduces the bias towards classifying unseen intents into seen ones. In a related, but orthogonal, line of research, the authors in (Ma et al, 2019;Gulyaev et al, 2020) addressed the problem of intent detection in the context of dialog state tracking where dialog state and conversation history are available in addition to an input utterance. In contrast, this work and the SOTA models we compare against in our experiments only consider an utterance without having access to any dialog state elements.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Generalized Zero-shot Intent Detection via Commonsense Knowledge

Siddique

Jamour

et al. 2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Identifying user intents from natural language utterances is a crucial step in conversational systems that has been extensively studied as a supervised classification problem. However, in practice, new intents emerge after deploying an intent detection model. Thus, these models should seamlessly adapt and classify utterances with both seen and unseen intents -unseen intents emerge after deployment and they do not have training data. The few existing models that target this setting rely heavily on the training data of seen intents and consequently overfit to these intents, resulting in a bias to misclassify utterances with unseen intents into seen ones. We propose RIDE: an intent detection model that leverages commonsense knowledge in an unsupervised fashion to overcome the issue of training data scarcity. RIDE computes robust and generalizable relationship meta-features that capture deep semantic relationships between utterances and intent labels; these features are computed by considering how the concepts in an utterance are linked to those in an intent label via commonsense knowledge. Our extensive experimental analysis on three widely-used intent detection benchmarks shows that relationship meta-features significantly improve the detection of both seen and unseen intents and that RIDE outperforms the state-of-the-art models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Generalized Zero-shot Intent Detection via Commonsense Knowledge

Siddique

Jamour

et al. 2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

show abstract

“…These methods define DST as either a classification problem or a generation problem. Motivated by the advances in reading comprehension [4], DST has been further formulated as a machine reading comprehension problem [13,14,30,31]. Other techniques such as pointer networks [56] and reinforcement learning [7,8,23] have also been applied to DST.…”

Section: Related Workmentioning

confidence: 99%

Slot Self-Attentive Dialogue State Tracking

Manotumruksa

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

An indispensable component in task-oriented dialogue systems is the dialogue state tracker, which keeps track of users' intentions in the course of conversation. The typical approach towards this goal is to fill in multiple pre-defined slots that are essential to complete the task. Although various dialogue state tracking methods have been proposed in recent years, most of them predict the value of each slot separately and fail to consider the correlations among slots. In this paper, we propose a slot self-attention mechanism that can learn the slot correlations automatically. Specifically, a slottoken attention is first utilized to obtain slot-specific features from the dialogue context. Then a stacked slot self-attention is applied on these features to learn the correlations among slots. We conduct comprehensive experiments on two multi-domain task-oriented dialogue datasets, including MultiWOZ 2.0 and MultiWOZ 2.1. The experimental results demonstrate that our approach achieves stateof-the-art performance on both datasets, verifying the necessity and effectiveness of taking slot correlations into consideration. CCS CONCEPTS• Computing methodologies → Natural language processing; Discourse, dialogue and pragmatics; Machine learning.

show abstract

“…Schema-guided modeling builds on work on building task-oriented dialogue systems that can generalize easily to new verticals using very little extra information, including for slot filling (Bapna et al 2017;Shah et al 2019;Liu et al 2020) and dialogue state tracking (Li et al 2021;Campagna et al 2020;Kumar et al 2020) among other tasks. More recent work has adopted the schema-guided paradigm (Ma et al 2019;Li, Xiong, and Cao 2020;Zhang et al 2021) and even extended the paradigm in functionality (Mosig, Mehri, and Kober 2020;Mehri and Eskenazi 2021). Lin et al (2021) and Cao and Zhang (2021) both investigate different natural language description styles for dialogue state tracking generalization.…”

Section: Related Workmentioning

confidence: 99%

SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems

Lee,

Gupta,

Rastogi

et al. 2021

Preprint

View full text Add to dashboard Cite

Zero/few-shot transfer to unseen services is a critical challenge in task-oriented dialogue research. The Schema-Guided Dialogue (SGD) dataset introduced a paradigm for enabling models to support an unlimited number of services without additional data collection or re-training through the use of schemas. Schemas describe service APIs in natural language, which models consume to understand the services they need to support. However, the impact of the choice of language in these schemas on model performance remains unexplored. We address this by releasing SGD-X, a benchmark for measuring the robustness of dialogue systems to linguistic variations in schemas. SGD-X extends the SGD dataset with crowdsourced variants for every schema, where variants are semantically similar yet stylistically diverse. We evaluate two dialogue state tracking models on SGD-X and observe that neither generalizes well across schema variations, measured by joint goal accuracy and a novel metric for measuring schema sensitivity. Furthermore, we present a simple modelagnostic data augmentation method to improve schema robustness and zero-shot generalization to unseen services.

show abstract

An End-to-End Dialogue State Tracking System with Machine Reading Comprehension and Wide & Deep Classification

Cited by 19 publications

References 16 publications

Generalized Zero-shot Intent Detection via Commonsense Knowledge

Generalized Zero-shot Intent Detection via Commonsense Knowledge

Slot Self-Attentive Dialogue State Tracking

SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems

Contact Info

Product

Resources

About