Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking

Campagna, G.; Foryciarz, Agata; Moradshahi, Mehrad; Lam, Monica S.

doi:10.18653/v1/2020.acl-main.12

Cited by 64 publications

(62 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They Kumar et al (2020) and Li et al (2021) follow the same experimental setup, and we also compare NeuralWOZ in the same experiment setup. Abstract Transaction Dialogue Model (ATDM) (Campagna et al, 2020), another method for synthesizing dialogue data, is another baseline for zero-shot domain transfer tasks we adopt. They use rules, abstract state transition, and templates to synthesize the dialogue, which is then fed into a model-based zero-shot learner.…”

Section: Zero-shot Domain Transfermentioning

confidence: 99%

“…Other studies on dialogue datasets use the user simulator-based data collection approaches (Schatzmann et al, 2007;Li et al, 2017;Bordes et al, 2017;Shah et al, 2018;Zhao and Eskenazi, 2018;Shah et al, 2018;Campagna et al, 2020). They define domain schema, rules, and dialogue templates to simulate user behavior under certain goals.…”

Section: Synthetic Dialogue Generationmentioning

confidence: 99%

“…We evaluate our method for zero-shot domain transfer task Campagna et al, 2020) to demonstrate the ability to generate corpus for unseen domains, when no prior training data exists. In dialogue state tracking (DST) task with MultiWOZ 2.1 (Eric et al, 2019), the synthetic data generated with NeuralWOZ achieves 4.4% point higher joint goal accuracy and 5.7% point higher zero-shot coverage than the existing baseline.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation

Kim¹,

Chang²,

Lee³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

We propose NeuralWOZ, a novel dialogue collection framework that uses model-based dialogue simulation. NeuralWOZ has two pipelined models, Collector and Labeler. Collector generates dialogues from (1) user's goal instructions, which are the user context and task constraints in natural language, and (2) system's API call results, which is a list of possible query responses for user requests from the given knowledge base. Labeler annotates the generated dialogue by formulating the annotation as a multiple-choice problem, in which the candidate labels are extracted from goal instructions and API call results. We demonstrate the effectiveness of the proposed method in the zero-shot domain transfer learning for dialogue state tracking. In the evaluation, the synthetic dialogue corpus generated from NeuralWOZ achieves a new state-of-theart with improvements of 4.4% point joint goal accuracy on average across domains, and improvements of 5.7% point of zero-shot coverage against the MultiWOZ 2.1 dataset. 1

show abstract

Section: Zero-shot Domain Transfermentioning

confidence: 99%

Section: Synthetic Dialogue Generationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation

Kim¹,

Chang²,

Lee³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…Dialogue diversity can be improved via enforcing diversity objective functions (such as maximize mutual information) in neural models (Li et al, 2016a;Baheti et al, 2018), perturbing language rules (Niu and Bansal, 2019) or environment parameters Ruiz et al, 2019), randomizing trajectory synthesis (Andrychowicz et al, 2017;Lu et al, 2019), selecting more diverse data contributors (Stasaski et al, 2020), and sampling trajectories from a diverse set of environments (Chua et al, 2018;Janner et al, 2019). For instance, Campagna et al augmented dialogue data using domain-independent transition rules and domain-specific ontology (Campagna et al, 2020). Niu and Bansal synthesized more diverse dialogue trajectories by choosing semantic-preserving language perturbations via RL (Niu and Bansal, 2019).…”

Section: Diversification In Dialoguesmentioning

confidence: 99%

High-Quality Dialogue Diversification by Intermittent Short Extension Ensembles

Tang¹,

Kulkarni²,

Yang³

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Many task-oriented dialogue systems use deep reinforcement learning (DRL) to learn policies that respond to the user appropriately and complete the tasks successfully. Training DRL agents with diverse dialogue trajectories prepare them well for rare user requests and unseen situations. One effective diversification method is to let the agent interact with a diverse set of learned user models. However, trajectories created by these artificial user models may contain generation errors, which can quickly propagate into the agent's policy. It is thus important to control the quality of the diversification and resist the noise. In this paper, we propose a novel dialogue diversification method for task-oriented dialogue systems trained in simulators. Our method, Intermittent Short Extension Ensemble (I-SEE), 1 constrains the intensity to interact with an ensemble of diverse user models and effectively controls the quality of the diversification. Evaluations on the Multiwoz dataset show that I-SEE successfully boosts the performance of several state-of-the-art DRL dialogue agents.

show abstract

“…Stateof-the-art methods, based on sequence-to-sequence neural networks, require large amounts of manually annotated data (Dong and Lapata, 2016;Jia and Liang, 2016). Various methods have been proposed to eliminate manually annotated data for new domains, using synthesis (Wang et al, 2015;Shah et al, 2018;Campagna et al, 2019;Xu et al, 2020a,b), transfer learning (Zhong et al, 2017;Herzig and Berant, 2018;Yu et al, 2018;, or a combination of both (Rastogi et al, 2019;Campagna et al, 2020). All these works focus mainly on the English language, and have not been applied to other languages.…”

Section: Related Workmentioning

confidence: 99%

Localizing Open-Ontology QA Semantic Parsers in a Day Using Machine Translation

Moradshahi¹,

Campagna²,

J.³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

We propose Semantic Parser Localizer (SPL), a toolkit that leverages Neural Machine Translation (NMT) systems to localize a semantic parser for a new language. Our methodology is to (1) generate training data automatically in the target language by augmenting machine-translated datasets with local entities scraped from public websites, (2) add a fewshot boost of human-translated sentences and train a novel XLMR-LSTM semantic parser, and (3) test the model on natural utterances curated using human translators.We assess the effectiveness of our approach by extending the current capabilities of Schema2QA, a system for English Question Answering (QA) on the open web, to 10 new languages for the restaurants and hotels domains. Our models achieve an overall test accuracy ranging between 61% and 69% for the hotels domain and between 64% and 78% for restaurants domain, which compares favorably to 69% and 80% obtained for English parser trained on gold English data and a few examples from validation set. We show our approach outperforms the previous state-of-theart methodology by more than 30% for hotels and 40% for restaurants with localized ontologies for the subset of languages tested.Our methodology enables any software developer to add a new language capability to a QA system for a new domain, leveraging machine translation, in less than 24 hours. Our code is released open-source. 1

show abstract

Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking

Cited by 64 publications

References 22 publications

NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation

NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation

High-Quality Dialogue Diversification by Intermittent Short Extension Ensembles

Localizing Open-Ontology QA Semantic Parsers in a Day Using Machine Translation

Contact Info

Product

Resources

About