Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging

Plank, Barbara; Agić, Željko

doi:10.18653/v1/d18-1061

Cited by 47 publications

(62 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Yu et al (2018b) investigated using character-based language models for NER in several languages but did not do any cross-lingual learning. Plank and Agić (2018) used cross-lingual embeddings, projected annotations, and dictionaries for zero-shot cross-lingual part-of-speech tagging.…”

Section: Related Workmentioning

confidence: 99%

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Schuster¹,

Gupta²,

Shah³

et al. 2019

Proceedings of the 2019 Conference of the North

205

231

View full text Add to dashboard Cite

One of the first steps in the utterance interpretation pipeline of many task-oriented conversational AI systems is to identify user intents and the corresponding slots. Since data collection for machine learning models for this task is time-consuming, it is desirable to make use of existing data in a high-resource language to train models in low-resource languages. However, development of such models has largely been hindered by the lack of multilingual training data. In this paper, we present a new data set of 57k annotated utterances in English (43k), Spanish (8.6k) and Thai (5k) across the domains weather, alarm, and reminder. We use this data set to evaluate three different cross-lingual transfer methods: (1) translating the training data, (2) using cross-lingual pre-trained embeddings, and (3) a novel method of using a multilingual machine translation encoder as contextual word representations. We find that given several hundred training examples in the the target language, the latter two methods outperform translating the training data. Further, in very low-resource settings, multilingual contextual word representations give better results than using cross-lingual static embeddings. We also compare the cross-lingual methods to using monolingual resources in the form of contextual ELMo representations and find that given just small amounts of target language data, this method outperforms all cross-lingual methods, which highlights the need for more sophisticated cross-lingual methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Schuster¹,

Gupta²,

Shah³

et al. 2019

Proceedings of the 2019 Conference of the North

205

231

View full text Add to dashboard Cite

show abstract

“…A common challenge in applying natural language processing (NLP) techniques to low-resource languages is the lack of training data in the languages in question. It has been demonstrated that through cross-lingual transfer, it is possible to leverage one or more similar high-resource languages to improve the performance on the low-resource languages in several NLP tasks, including machine score(L tf,1 , L tk ) score (L tf,2 , L tk translation (Zoph et al, 2016;Johnson et al, 2017;Nguyen and Chiang, 2017;Neubig and Hu, 2018), parsing (Täckström et al, 2012;Ammar et al, 2016;Ahmad et al, 2019;, partof-speech or morphological tagging (Täckström et al, 2013;Cotterell and Heigold, 2017;Malaviya et al, 2018;Plank and Agić, 2018), named entity recognition (Zhang et al, 2016;Mayhew et al, 2017;Xie et al, 2018), and entity linking (Tsai and Roth, 2016;Rijhwani et al, 2019). There are many methods for performing this transfer, including joint training (Ammar et al, 2016;Tsai and Roth, 2016;Cotterell and Heigold, 2017;Johnson et al, 2017;Malaviya et al, 2018), annotation projection (Täckström et al, 2012;Täckström et al, 2013;Zhang et al, 2016;Plank and Agić, 2018), fine-tuning (Zoph et al, 2016;Neubig and Hu, 2018), data augmentation (Mayhew et al, 2017), or zero-shot transfer (Ahmad et al, 2019;Xie et al, 2018;Neubig and Hu, 2018;Rijhwani et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Choosing Transfer Languages for Cross-Lingual Learning

Lin¹,

Chen²,

Lee³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

102

View full text Add to dashboard Cite

Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on lowresource languages. However, given a particular task language, it is not clear which language to transfer from, and the standard strategy is to select languages based on ad hoc criteria, usually the intuition of the experimenter. Since a large number of features contribute to the success of cross-lingual transfer (including phylogenetic similarity, typological properties, lexical overlap, or size of available data), even the most enlightened experimenter rarely considers all these factors for the particular task at hand. In this paper, we consider this task of automatically selecting optimal transfer languages as a ranking problem, and build models that consider the aforementioned features to perform this prediction. In experiments on representative NLP tasks, we demonstrate that our model predicts good transfer languages much better than ad hoc baselines considering single features in isolation, and glean insights on what features are most informative for each different NLP tasks, which may inform future ad hoc selection even without use of our method. 1 * Equal contribution 1 Code, data, and pre-trained models are available at

show abstract

“…In the multi-source projection experiments, our criteria for filtering is based on whether the sentence was present across all target treebanks and more sophisticated approaches could be used to select better training instances as in Plank and Agić (2018).…”

Section: Resultsmentioning

confidence: 99%

Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A Faroese Case Study

Barry¹,

Wagner²,

Foster³

2019

Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)

View full text Add to dashboard Cite

Cross-lingual dependency parsing involves transferring syntactic knowledge from one language to another. It is a crucial component for inducing dependency parsers in low-resource scenarios where no training data for a language exists. Using Faroese as the target language, we compare two approaches using annotation projection: first, projecting from multiple monolingual source models; second, projecting from a single polyglot model which is trained on the combination of all source languages. Furthermore, we reproduce multisource projection (Tyers et al., 2018), in which dependency trees of multiple sources are combined. Finally, we apply multi-treebank modelling to the projected treebanks, in addition to or alternatively to polyglot modelling on the source side. We find that polyglot training on the source languages produces an overall trend of better results on the target language but the single best result for the target language is obtained by projecting from monolingual source parsing models and then training multi-treebank POS tagging and parsing models on the target side.

show abstract

Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging

Cited by 47 publications

References 22 publications

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Choosing Transfer Languages for Cross-Lingual Learning

Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A Faroese Case Study

Contact Info

Product

Resources

About