Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack

Abstract

Zero-shot slot filling has widely arisen to cope with data scarcity in target domains. However, previous approaches often ignore constraints between slot value representation and related slot description representation in the latent space and lack enough model robustness. In this paper, we propose a Contrastive Zero-Shot Learning with Adversarial Attack (CZSL-Adv) method for the cross-domain slot filling. The contrastive loss aims to map slot value contextual representations to the corresponding slot description representations. And we introduce an adversarial attack training strategy to improve model robustness. Experimental results show that our model significantly outperforms state-of-the-art baselines under both zero-shot and few-shot settings.

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

“…In the second stage, our model encodes the slot entity and predicts the label for it by calculating the similarity with the slot prototypes in the label semantic space. Unlike the previous works (Liu et al, 2020b;He et al, 2020c) which directly utilize the slot name embedding as slot prototypes, we introduce Prototypical Contrastive learning and Label Confusion strategies (PCLC) strategies to dynamically refine the constraint relationship between slot prototypes in the semantic space, as shown in Fig 2 . In the training procedure, we use an MLP layer to encode the original slot name embedding. So we can obtain a dynamically updated slot prototype matrix.…”

“…• Contrastive Zero-Shot Learning with Adversarial Attack (CZSL-Adv) A method proposed by (He et al, 2020c) based on Coach, which utilizes contrastive learning and adversarial attacks to improve the performance and robustness of the framework. Implementation Details We follow the setup of (Liu et al, 2020b), selecting one domain as the target domain at a time, and use 500 samples in this domain as a validation set, the rest as a test set.…”

“…The main drawback is that the model possibly predicts multiple slot types for one entity span. To avoid the above problem, (Liu et al, 2020b,a;He et al, 2020c) decompose the slot filling task into two stages. First, all slot entities in the utterances are identified by the coarse-grained binary sequence labeling model.…”

Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System

“…for pre-defined slot types, using character embedding (Liang et al, 2017a), copy mechanism (Zhao and Feng, 2018), few/zero-shot learning (Hu et al, 2019;He et al, 2020e;Shah et al, 2019), transfer learning (Chen and Moschitti, 2019;He et al, 2020c,b) and background knowledge (Yang and Mitchell, 2017;He et al, 2020d), etc. Compared to OOV recognition, our proposed novel slot detection task focuses on detecting unknown slot types, not just unseen values.…”

“…NSD aims to discover potential new or out-of-domain entity types to strengthen the capability of a dialogue system based on in-domain precollected training data. There are two aspects in the previous work related to NSD, out-of-vocabulary (OOV) recognition (Liang et al, 2017a;Zhao and Feng, 2018;Hu et al, 2019;He et al, 2020c,d;Yan et al, 2020;He et al, 2020e) and out-of-domain (OOD) intent detection (Lin and Xu, 2019;Larson et al, 2019;Xu et al, 2020a;Zeng et al, 2021b,a) 1: Comparison between slot filling and novel slot detection. In the novel slot detection labels, we consider "album" as an unknown slot type that is out of the scope of the pre-defined slot set.…”