Out-of-domain Detection based on Generative Adversarial Network

Ryu, Seon Young; Koo, Sangjun; Yu, Hwanjo; Lee, Gary Geunbae

doi:10.18653/v1/d18-1077

Cited by 53 publications

(55 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, the relative performance gain for FPR95 and FPR90 on the OSQ dataset 3) The Cont. GAN baseline proposed in [16] only performs slightly better than random guess. This shows that using GAN to generate continuous features for OOD samples is not helpful to improve the OOD detection performance, and the discriminator learned in the adversarial training process cannot be directly used as the OOD detector.…”

Section: E Effects Of Generated Pseudo Ood Utterancesmentioning

confidence: 93%

“…There are two closely relevant studies from Ryu et al [16] and Lee et al [14]. These studies utilize a GAN based generator to produce OOD samples when building the OOD detector.…”

Section: Related Workmentioning

confidence: 99%

“…7) Cont. GAN [16]: A GAN-based model is trained to generate continuous OOD features to mimic features extracted from IND samples. The confidence of the discriminator is used as the detection score.…”

Section: Baselinesmentioning

confidence: 99%

See 2 more Smart Citations

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

Zheng

Chen

Huang

2020

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Natural Language Understanding (NLU) is a vital component of dialogue systems, and its ability to detect Out-of-Domain (OOD) inputs is critical in practical applications, since the acceptance of the OOD input that is unsupported by the current system may lead to catastrophic failure. However, most existing OOD detection methods rely heavily on manually labeled OOD samples and cannot take full advantage of unlabeled data. This limits the feasibility of these models in practical applications.In this paper, we propose a novel model to generate highquality pseudo OOD samples that are akin to IN-Domain (IND) input utterances, and thereby improves the performance of OOD detection. To this end, an autoencoder is trained to map an input utterance into a latent code. and the codes of IND and OOD samples are trained to be indistinguishable by utilizing a generative adversarial network. To provide more supervision signals, an auxiliary classifier is introduced to regularize the generated OOD samples to have indistinguishable intent labels. Experiments show that these pseudo OOD samples generated by our model can be used to effectively improve OOD detection in NLU. Besides, we also demonstrate that the effectiveness of these pseudo OOD data can be further improved by efficiently utilizing unlabeled data.

show abstract

Section: E Effects Of Generated Pseudo Ood Utterancesmentioning

confidence: 93%

“…There are two closely relevant studies from Ryu et al [16] and Lee et al [14]. These studies utilize a GAN based generator to produce OOD samples when building the OOD detector.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

Zheng

Chen

Huang

2020

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Out-of-Domain Detection Existing methods often formulate the OOD task as a one-class classification problem, then use appropriate methods to solve it (e.g., one-class SVM (Schölkopf et al, 2001) and one-class DL-based classifiers (Ruff et al, 2018;Manevitz and Yousef, 2007). A group of researchers also proposed an auto-encoderbased approach and its variation to tackle OOD tasks (Ryu et al, 2017(Ryu et al, , 2018. Recently, a few papers have investigated ID classification and OOD detection simultaneously (Kim and Kim, 2018;Shu et al, 2017), but they fail in a low resource setting.…”

Section: Related Workmentioning

confidence: 99%

Out-of-Domain Detection for Low-Resource Text Classification Tasks

Tan

Wang

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Out-of-domain (OOD) detection for lowresource text classification is a realistic but understudied task. The goal is to detect the OOD cases with limited in-domain (ID) training data, since we observe that training data is often insufficient in machine learning applications. In this work, we propose an OODresistant Prototypical Network to tackle this zero-shot OOD detection and few-shot ID classification task. Evaluation on real-world datasets show that the proposed solution outperforms state-of-the-art methods in zero-shot OOD detection task, while maintaining a competitive performance on ID classification task.

show abstract

“…Many techniques have been proposed for OOD detection based on traditional classification algorithms, such as OCSVM [ 41 , 43 , 44 , 45 ], KNN [ 15 ], and Kmeans [ 16 ]. Recently, deep learning-based OOD-detection techniques have been proposed [ 17 , 19 , 20 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 ], some of which we now highlight.…”

Section: Background and Related Workmentioning

confidence: 99%

Out-of-Distribution Detection of Human Activity Recognition with Smartwatch Inertial Sensors

Boyer

Burns

Whyne

2021

Sensors

View full text Add to dashboard Cite

Out-of-distribution (OOD) in the context of Human Activity Recognition (HAR) refers to data from activity classes that are not represented in the training data of a Machine Learning (ML) algorithm. OOD data are a challenge to classify accurately for most ML algorithms, especially deep learning models that are prone to overconfident predictions based on in-distribution (IIN) classes. To simulate the OOD problem in physiotherapy, our team collected a new dataset (SPARS9x) consisting of inertial data captured by smartwatches worn by 20 healthy subjects as they performed supervised physiotherapy exercises (IIN), followed by a minimum 3 h of data captured for each subject as they engaged in unrelated and unstructured activities (OOD). In this paper, we experiment with three traditional algorithms for OOD-detection using engineered statistical features, deep learning-generated features, and several popular deep learning approaches on SPARS9x and two other publicly-available human activity datasets (MHEALTH and SPARS). We demonstrate that, while deep learning algorithms perform better than simple traditional algorithms such as KNN with engineered features for in-distribution classification, traditional algorithms outperform deep learning approaches for OOD detection for these HAR time series datasets.

show abstract

Out-of-domain Detection based on Generative Adversarial Network

Cited by 53 publications

References 11 publications

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

Out-of-Domain Detection for Low-Resource Text Classification Tasks

Out-of-Distribution Detection of Human Activity Recognition with Smartwatch Inertial Sensors

Contact Info

Product

Resources

About