Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking

Lin, Zhaojiang; Liu, Bing; Moon, Seungwhan; Crook, Paul A.; Zhou, Zhenpeng; Yu, Zhou; Madotto, Andrea; Cho, Eunjoon; Subba, Rajen

doi:10.48550/arxiv.2105.04222

Cited by 5 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here we compare with the baseline models that can predict dialogue state on unseen domains: SGDbaseline , TransferQA (Lin et al, 2021a), SDM-DST , SUMBT (Lee et al, 2019), SimpleTOD (Hosseini-Asl et al, 2020), T5DST (Lin et al, 2021b) and D3ST method (Zhao et al, 2022). Tables 2 and 3 highlight the exceptional performance of our approach in zero-shot cross-domain DST.…”

Section: Resultsmentioning

confidence: 99%

Towards LLM-driven Dialogue State Tracking

Feng,

Lu,

Liu

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Dialogue State Tracking (DST) is of paramount importance in ensuring accurate tracking of user goals and system actions within taskoriented dialogue systems. The emergence of large language models (LLMs) such as GPT3 and ChatGPT has sparked considerable interest in assessing their efficacy across diverse applications. In this study, we conduct an initial examination of ChatGPT's capabilities in DST. Our evaluation uncovers the exceptional performance of ChatGPT in this task, offering valuable insights to researchers regarding its capabilities and providing useful directions for designing and enhancing dialogue systems. Despite its impressive performance, ChatGPT has significant limitations including its closedsource nature, request restrictions, raising data privacy concerns, and lacking local deployment capabilities. To address these concerns, we present LDST, an LLM-driven DST framework based on smaller, open-source foundation models. By utilizing a novel domain-slot instruction tuning method, LDST achieves performance on par with ChatGPT. Comprehensive evaluations across three distinct experimental settings, we find that LDST exhibits remarkable performance improvements in both zero-shot and few-shot setting compared to previous SOTA methods. The source code 1 is provided for reproducibility.

show abstract

Section: Resultsmentioning

confidence: 99%

Towards LLM-driven Dialogue State Tracking

Feng,

Lu,

Liu

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…T5DST (Lin et al, 2021b) A slot description enhanced approach for zero-shot & few-shot crossdomain DST based on T5.…”

Section: A Appendixmentioning

confidence: 99%

Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Tracking

Wu,

Dong,

2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Zero-shot Dialogue State Tracking (DST) addresses the challenge of acquiring and annotating task-oriented dialogues, which can be timeconsuming and costly. However, DST extends beyond simple slot-filling and requires effective updating strategies for tracking dialogue state as conversations progress. In this paper, we propose ParsingDST, a new In-Context Learning (ICL) method, to introduce additional intricate updating strategies in zero-shot DST. Our approach reformulates the DST task by leveraging powerful Large Language Models (LLMs) and translating the original dialogue text to JSON through semantic parsing as an intermediate state. We also design a novel framework that includes more modules to ensure the effectiveness of updating strategies in the text-to-JSON process. Experimental results demonstrate that our approach outperforms existing zero-shot DST methods on MultiWOZ, exhibiting significant improvements in Joint Goal Accuracy (JGA) and slot accuracy compared to existing ICL methods.

show abstract

“…Other permutations of these modules have also been explored. For instance, several systems [52][53][54][55] remove the NLU module and replace it with only a DST. In contrast, others build a very strong NLU module and ignore the DST [56].…”

Section: Module-based Task-oriented Dialogue Systemsmentioning

confidence: 99%

Taming the Beast: Learning to Control Neural Conversational Models

Madotto¹

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This thesis investigates the controllability of deep learning-based, end-to-end, generative dialogue systems in both task-oriented and chit-chat scenarios. In particular, we study the different aspects of controlling generative dialogue systems, including controlling styles and topics and continuously adding and combining dialogue skills.In the three decades since the first dialogue system was commercialized, the basic architecture of such systems has remained substantially unchanged, consisting of four pipelined basic components, namely, natural language understanding (NLU), dialogue state tracking (DST), a dialogue manager (DM) and natural language generation (NLG). The dialogue manager, which is the critical component of the modularized system, controls the response content and style.This module is usually programmed by rules and is designed to be highly controllable and easily extendable.With the emergence of powerful "deep learning" architectures, end-to-end generative dialogue systems have been proposed to optimize overall system performance and simplify training.However, these systems cannot be easily controlled and extended as the modularized dialogue manager can. This is because a single neural system is used, which is usually a large pre-trained language model (e.g., GPT-2), and thus it is hard to surgically change desirable attributes (e.g., style, topics, etc.). More importantly, uncontrollable dialogue systems can generate offensive and even toxic responses.Therefore, in this thesis, we study controllable methods for end-to-end generative dialogue systems in task-oriented and chit-chat scenarios. Throughout the chapters, we describe 1) how to control the style and topics of chit-chat models, 2) how to continuously control and extend x task-oriented dialogue systems, and 3) how to compose and control multi-skill dialogue models.To elaborate, we firstly propose a residual adapter model to control style and topics in conversational models such as DialoGPT, Meena, and Blender-Bot. Our proposed model adds less than 1.5% task-specific parameters per style/topic, making it deployable for online systems. We run a comprehensive automatic and human evaluation to show controllability in the response generation in terms of style and topics, without losing fluency without requiring dialogue-specific datasets.Secondly, we propose a highly controllable architectural method based on residual adapters for continuous update of task-oriented dialogue systems with new features based on the user's needs, e.g., adding new slots and intents or even completely new domains. Moreover, we analyze the trade-off between performance, number-of-parameters, and episodic memory sizes in other methods (regularization, rehearsal, architectural).Finally, we propose a novel theoretical framework to control the end-to-end dialogue model with multiple composable and control skills. We empirically show the effectiveness of using specialized parameters in combined chit-chat and task-oriented datasets.xi

show abstract

Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking

Cited by 5 publications

References 25 publications

Towards LLM-driven Dialogue State Tracking

Towards LLM-driven Dialogue State Tracking

Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Tracking

Taming the Beast: Learning to Control Neural Conversational Models

Contact Info

Product

Resources

About