Learning Multilingual Meta-Embeddings for Code-Switching Named Entity Recognition

Winata, Genta Indra; Lin, Zhaojiang; Fung, Pascale

doi:10.18653/v1/w19-4320

Cited by 29 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another method that uses multilingual meta-embeddings for code-switching NER was proposed by Genta Indra Winata et al [48]. In this work, the authors proposed multilingual meta-embeddings which is an effective method to learn multilingual representations by using monolingual pretrained embeddings [48]. This paper claims that their proposed method achieves state-of-the-art performance in a multilingual setting and has the generalization ability.…”

Section: Ner In English and Other Resource Heavy Languagesmentioning

confidence: 96%

“…Furthermore, they have constructed a baseline NER system using deep neural networks and word embedding for Arabic-English CM text and enhanced it using a pooling technique [47]. Another method that uses multilingual meta-embeddings for code-switching NER was proposed by Genta Indra Winata et al [48]. In this work, the authors proposed multilingual meta-embeddings which is an effective method to learn multilingual representations by using monolingual pretrained embeddings [48].…”

Section: Ner In English and Other Resource Heavy Languagesmentioning

confidence: 99%

See 1 more Smart Citation

A Framework for Named Entity Recognition for Malayalam—A Comparison of Different Deep Learning Architectures

Rajimol¹,

Anoop²

2020

NLPRE

View full text Add to dashboard Cite

Information extraction (IE) is the process of extracting relevant and useful patterns or information from unstructured data. Named entity recognition (NER) is a subtask of IE that identifies entities from unstructured text documents and organize them into different predefined categories such as person, location, organization, number, date, etc. NER is considered to be one of the important steps in natural language processing which may find direct applications in areas such as question answering (QA), entity linking, and co-reference resolution, to name a few. NER systems perform comparatively well in high-resource languages such as English but there is a lack of well-developed NER systems for low-resource languages such as Malayalam, which is an Indic language spoken in the state of Kerala, India. This work is an approach in this direction which makes use of deep learning (DL) techniques for developing a NER system for Malayalam. We have compared different DL approaches such as recurrent neural networks, gated recurrent unit, long short-term memory, and bi-directional long short-term memory and found that DL based approaches significantly outperform traditional shallow-learning based -approaches for NER. When compared with some state-of-the-art approaches our proposed framework is found to be outperforming in terms of precision, recall, and F-measure and could achieve an improved precision of 7.89% and 8.92% of F-measure.

show abstract

Section: Ner In English and Other Resource Heavy Languagesmentioning

confidence: 96%

Section: Ner In English and Other Resource Heavy Languagesmentioning

confidence: 99%

A Framework for Named Entity Recognition for Malayalam—A Comparison of Different Deep Learning Architectures

Rajimol¹,

Anoop²

2020

NLPRE

View full text Add to dashboard Cite

show abstract

“…(Xiao et al, 2020; applied meta-learning to solve the multilingual lowresource speech recognition problem. (Winata et al, 2019) also used MAML to adapt models to unseen accents on speech recognition. (Indurthi et al, 2019) adopted meta-learning algorithms to perform speech translation on speech-transcript paired low-resource data.…”

Section: Related Workmentioning

confidence: 99%

Multi-accent Speech Separation with One Shot Learning

Huang¹,

Wu²,

Lee³

2021

Proceedings of the 1st Workshop on Meta Learning and Its Applications to Natural Language Processing

View full text Add to dashboard Cite

Speech separation is a problem in the field of speech processing that has been studied in full swing recently. However, there has not been much work studying a multi-accent speech separation scenario. Unseen speakers with new accents and noise aroused the domain mismatch problem which cannot be easily solved by conventional joint training methods. Thus, we applied MAML and FOMAML to tackle this problem and obtained higher average Si-SNRi values than joint training on almost all the unseen accents. This proved that these two methods do have the ability to generate well-trained parameters for adapting to speech mixtures of new speakers and accents. Furthermore, we found out that FOMAML obtains similar performance compared to MAML while saving a lot of time.

show abstract

“…Wang et al (2018) use a different attention method for NER, which is based on a gated cell that learns to choose appropriate monolingual embeddings according to the input text. Recently, Winata et al (2019) proposed multilingual meta embeddings (MME) combined with self-attention (Vaswani et al, 2017). Their method establishes a state of the art on Spanish-English NER by heavily relying on monolingual embeddings for every language in the code-switched text.…”

Section: Related Workmentioning

confidence: 99%

From English to Code-Switching: Transfer Learning with Strong Morphological Clues

Aguilar¹,

Solorio²

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Linguistic Code-switching (CS) is still an understudied phenomenon in natural language processing. The NLP community has mostly focused on monolingual and multilingual scenarios, but little attention has been given to CS in particular. This is partly because of the lack of resources and annotated data, despite its increasing occurrence in social media platforms. In this paper, we aim at adapting monolingual models to code-switched text in various tasks. Specifically, we transfer English knowledge from a pre-trained ELMo model to different code-switched language pairs (i.e., Nepali-English, Spanish-English, and Hindi-English) using the task of language identification. Our method, CS-ELMo, is an extension of ELMo with a simple yet effective position-aware attention mechanism inside its character convolutions. We show the effectiveness of this transfer learning step by outperforming multilingual BERT and homologous CS-unaware ELMo models and establishing a new state of the art in CS tasks, such as NER and POS tagging. Our technique can be expanded to more English-paired code-switched languages, providing more resources to the CS community.

show abstract

Learning Multilingual Meta-Embeddings for Code-Switching Named Entity Recognition

Cited by 29 publications

References 24 publications

A Framework for Named Entity Recognition for Malayalam—A Comparison of Different Deep Learning Architectures

A Framework for Named Entity Recognition for Malayalam—A Comparison of Different Deep Learning Architectures

Multi-accent Speech Separation with One Shot Learning

From English to Code-Switching: Transfer Learning with Strong Morphological Clues

Contact Info

Product

Resources

About