Meta Learning and Its Applications to Natural Language Processing

Lee, Hung-yi; Vu, Ngoc Thang; Li, Shang-Wen

doi:10.18653/v1/2021.acl-tutorials.3

Cited by 7 publications

(4 citation statements)

References 92 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meta-Learning for Semantic Parsing A variety of NLP applications have adopted metalearning in zero-and few-shot learning scenarios as a method of explicitly training for generalization (Lee et al, 2021;Hedderich et al, 2021). Within semantic parsing, there has been increasing interest in cross-database generalization, motivated by datasets such as Spider (Yu et al, 2018) requiring navigation of unseen databases (Herzig and Berant, 2017;Suhr et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Meta-Learning a Cross-lingual Manifold for Semantic Parsing

Sherborne

Lapata

2023

Transactions of the Association for Computational Linguistics

View full text Add to dashboard Cite

Localizing a semantic parser to support new languages requires effective cross-lingual generalization. Recent work has found success with machine-translation or zero-shot methods, although these approaches can struggle to model how native speakers ask questions. We consider how to effectively leverage minimal annotated examples in new languages for few-shot cross-lingual semantic parsing. We introduce a first-order meta-learning algorithm to train a semantic parser with maximal sample efficiency during cross-lingual transfer. Our algorithm uses high-resource languages to train the parser and simultaneously optimizes for cross-lingual generalization to lower-resource languages. Results across six languages on ATIS demonstrate that our combination of generalization steps yields accurate semantic parsers sampling ≤10% of source training data in each new language. Our approach also trains a competitive model on Spider using English with generalization to Chinese similarly sampling ≤10% of training data.1

show abstract

Section: Related Workmentioning

confidence: 99%

Meta-Learning a Cross-lingual Manifold for Semantic Parsing

Sherborne

Lapata

2023

Transactions of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…built between all users, and then, it is personalized for each client using their data (Kulkarni et al, 2020;Schneider and Vlachos, 2019;Lee et al, 2021). In such cases, each user has either an entirely separate model, or additional personal parameters, causing significant overheads, both in terms of storage of the large models, and the computation complexity of training separate models for each user.…”

Section: Random User Identifiermentioning

confidence: 99%

UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis

Mireshghallah¹,

Shrivastava²,

Shokouhi³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Global models are typically trained to be as generalizable as possible. Invariance to the specific user is considered desirable since models are shared across multitudes of users. However, these models are often unable to produce personalized responses for individual users, based on their data. Contrary to widely-used personalization techniques based on few-shot and meta-learning, we propose UserIdentifier, a novel scheme for training a single shared model for all users. Our approach produces personalized responses by prepending a fixed, user-specific non-trainable string (called "user identifier") to each user's input text. Unlike prior work, this method doesn't need any additional model parameters, any extra rounds of personal few-shot learning, or any change made to the vocabulary. We empirically study different types of user identifiers (numeric, alphanumeric, and also randomly generated) and demonstrate that, surprisingly, randomly generated user identifiers outperform the prefixtuning based state-of-the-art approach by up to 13%, on a suite of sentiment analysis datasets.

show abstract

“…English + Indic Train: This approach combines approaches (1) and (2). The model is first pre-finetuned (Lee et al, 2021;Aghajanyan et al, 2021) on English XNLI data and then finetuned on Indic language of IN-DICXNLI data.…”

Section: Training-evaluation Strategiesmentioning

confidence: 99%

IndicXNLI: Evaluating Multilingual Inference for Indian Languages

Aggarwal¹,

Gupta²,

Kunchukuttan³

2022

Preprint

View full text Add to dashboard Cite

While Indic NLP has made rapid advances recently in terms of the availability of corpora and pre-trained models, benchmark datasets on standard NLU tasks are limited. To this end, we introduce INDICXNLI, an NLI dataset for 11 Indic languages. It has been created by high-quality machine translation of the original English XNLI dataset and our analysis attests to the quality of INDICXNLI. By finetuning different pre-trained LMs on this IN-DICXNLI, we analyze various cross-lingual transfer techniques with respect to the impact of the choice of language models, languages, multi-linguality, mix-language input, etc. These experiments provide us with useful insights into the behaviour of pre-trained models for a diverse set of languages.

show abstract

Meta Learning and Its Applications to Natural Language Processing

Cited by 7 publications

References 92 publications

Meta-Learning a Cross-lingual Manifold for Semantic Parsing

Meta-Learning a Cross-lingual Manifold for Semantic Parsing

UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis

IndicXNLI: Evaluating Multilingual Inference for Indian Languages

Contact Info

Product

Resources

About