New Transfer Learning Techniques for Disparate Label Sets

Kim, Young–Bum; Stratos, Karl; Sarikaya, Ruhi; Jeong, Minwoo

doi:10.3115/v1/p15-1046

Cited by 55 publications

(58 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An intent is defined as the type of content the user is seeking. This task is part of the spoken language understanding problem (Li et al, 2009;Tur and De Mori, 2011;Kim et al, 2015c;Mesnil et al, 2015;Kim et al, 2015a;Xu and Sarikaya, 2014;Kim et al, 2015b;Kim et al, 2015d). The amount of training data we used ranges from 12k to 120k (in number of queries) across different domains, the test data was from 2k to 20k.…”

Section: Methodsmentioning

confidence: 99%

Scalable Semi-Supervised Query Classification Using Matrix Sketching

Kim

Stratos

Sarikaya

2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

Self Cite

View full text Add to dashboard Cite

The enormous scale of unlabeled text available today necessitates scalable schemes for representation learning in natural language processing. For instance, in this paper we are interested in classifying the intent of a user query. While our labeled data is quite limited, we have access to virtually an unlimited amount of unlabeled queries, which could be used to induce useful representations: for instance by principal component analysis (PCA). However, it is prohibitive to even store the data in memory due to its sheer size, let alone apply conventional batch algorithms. In this work, we apply the recently proposed matrix sketching algorithm to entirely obviate the problem with scalability (Liberty, 2013). This algorithm approximates the data within a specified memory bound while preserving the covariance structure necessary for PCA. Using matrix sketching, we significantly improve the user intent classification accuracy by leveraging large amounts of unlabeled queries.

show abstract

Section: Methodsmentioning

confidence: 99%

Scalable Semi-Supervised Query Classification Using Matrix Sketching

Kim

Stratos

Sarikaya

2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The closest prior work by Kim et al (2015) address a sequential labeling problem in NLU where the fine grained label sets across domains differ. However, they assume that there exists a bijective mapping between the coarse and fine-grained label sets across domains.…”

Section: Multi-task Learningmentioning

confidence: 99%

Cross-domain Text Classification with Multiple Domains and Disparate Label Sets

Bhatt

Sinha

Roy

2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Advances in transfer learning have let go the limitations of traditional supervised machine learning algorithms for being dependent on annotated training data for training new models for every new domain. However, several applications encounter scenarios where models need to transfer/adapt across domains when the label sets vary both in terms of count of labels as well as their connotations. This paper presents first-of-its-kind transfer learning algorithm for cross-domain classification with multiple source domains and disparate label sets. It starts with identifying transferable knowledge from across multiple domains that can be useful for learning the target domain task. This knowledge in the form of selective labeled instances from different domains is congregated to form an auxiliary training set which is used for learning the target domain task. Experimental results validate the efficacy of the proposed algorithm against strong baselines on a real world social media and the 20 Newsgroups datasets.

show abstract

“…In fact, CCA has recently been adapted to learning latent word representations in an interesting way: by dividing each word position into a token view (which only sees surrounding context) and a type view (which only sees the word itself) and performing a CCA between these two views (Dhillon et al, 2012;Stratos et al, 2014;Stratos et al, 2015;Kim et al, 2015c). CCA is also used to induce label representations (Kim et al, 2015d) and lexicon representations (Kim et al, 2015b).…”

Section: Canonical Correlation Analysis (Cca)mentioning

confidence: 99%

Part-of-speech Taggers for Low-resource Languages using CCA Features

Kim

Snyder

Sarikaya

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

In this paper, we address the challenge of creating accurate and robust partof-speech taggers for low-resource languages. We propose a method that leverages existing parallel data between the target language and a large set of resourcerich languages without ancillary resources such as tag dictionaries. Crucially, we use CCA to induce latent word representations that incorporate cross-genre distributional cues, as well as projected tags from a full array of resource-rich languages. We develop a probability-based confidence model to identify words with highly likely tag projections and use these words to train a multi-class SVM using the CCA features. Our method yields average performance of 85% accuracy for languages with almost no resources, outperforming a state-of-the-art partiallyobserved CRF model.

show abstract

New Transfer Learning Techniques for Disparate Label Sets

Cited by 55 publications

References 26 publications

Scalable Semi-Supervised Query Classification Using Matrix Sketching

Scalable Semi-Supervised Query Classification Using Matrix Sketching

Cross-domain Text Classification with Multiple Domains and Disparate Label Sets

Part-of-speech Taggers for Low-resource Languages using CCA Features

Contact Info

Product

Resources

About