Dependency Parsing of Code-Switching Data with Cross-Lingual Feature
            Representations

Partanen, Niko; Lim, KyungTae; Rießler, Michael; Poibeau, Thierry

doi:10.18653/v1/w18-0201

“…This means that NLP experiments often use annotations that are too coarse to be linguistically informative with regard to C-S. Constraint-free theories (Mahootian and Santorini, 1996;MacSwan, 2000) hold that nothing restricts switching apart from the grammatical requirements of the contributing languages. Testing such theories in NLP experiments would require syntactically parsed corpora that are rare for mixed language data (Partanen et al, 2018). In sum, working together, theoretical and computational linguists could create better tools for processing C-S than those currently available.…”

Section: Competing Models Of C-smentioning

confidence: 99%

A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies

Doğruöz¹,

Sitaram²,

Bullock³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

The analysis of data in which multiple languages are represented has gained popularity among computational linguists in recent years. So far, much of this research focuses mainly on the improvement of computational methods and largely ignores linguistic and social aspects of C-S discussed across a wide range of languages within the long-established literature in linguistics. To fill this gap, we offer a survey of code-switching (C-S) covering the literature in linguistics with a reflection on the key issues in language technologies. From the linguistic perspective, we provide an overview of structural and functional patterns of C-S focusing on the literature from European and Indian contexts as highly multilingual areas. From the language technologies perspective, we discuss how massive language models fail to represent diverse C-S types due to lack of appropriate training data, lack of robust evaluation benchmarks for C-S (across multilingual situations and types of C-S) and lack of end-toend systems that cover sociolinguistic aspects of C-S as well. Our survey will be a step towards an outcome of mutual benefit for computational scientists and linguists with a shared interest in multilingualism and C-S.

show abstract

“…CS benchmarks: From one of the recent surveys (Sitaram et al, 2019), linguistic CS has been studied in the context of many NLP tasks including language identification (Solorio et al, 2014) (Bali et al, 2014), POS tagging (Das, 2016), NER (Aguilar et al, 2019), parsing (Partanen et al, 2018), sentiment analysis (Vilares et al, 2015), and question answering ) (Raghavi et al, 2015. Many CS datasets have been made available through the shared-task series FIRE (Roy et al, 2013) and CALCS (Aguilar et al, 2018), which have focused mostly on core NLP tasks.…”

Section: Related Workmentioning

confidence: 99%

Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching

Chopra¹,

Rallabandi²,

Black³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Code-switching (CS), a ubiquitous phenomenon due to the ease of communication it offers in multilingual communities still remains an understudied problem in language processing. The primary reasons behind this are: (1) minimal efforts in leveraging large pretrained multilingual models, and (2) the lack of annotated data. The distinguishing case of low performance of multilingual models in CS is the intra-sentence mixing of languages leading to switch points. We first benchmark two sequence labeling tasks -POS and NER on 4 different language pairs with a suite of pretrained models to identify the problems and select the best performing model, char-BERT, among them (addressing (1)). We then propose a self training method to repurpose the existing pretrained models using a switch-point bias by leveraging unannotated data (addressing (2)). We finally demonstrate that our approach performs well on both tasks by reducing the gap between the switch point performance while retaining the overall performance on two distinct language pairs in both the tasks. We plan to release our models and the code for all our experiments.

show abstract

“…CS benchmarks: From one of the recent surveys (Sitaram et al, 2019), linguistic CS has been studied in the context of many NLP tasks including language identification (Solorio et al, 2014) (Bali et al, 2014), POS tagging (Das, 2016), NER (Aguilar et al, 2019), parsing (Partanen et al, 2018), sentiment analysis (Vilares et al, 2015), and question answering ) (Raghavi et al, 2015. Many CS datasets have been made available through the shared-task series FIRE (Roy et al, 2013) and CALCS (Aguilar et al, 2018), which have focused mostly on core NLP tasks.…”

Section: Related Workmentioning

confidence: 99%

Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching

Chopra¹,

Rallabandi²,

Black³

et al. 2021

Preprint

0

View full text Add to dashboard Cite

Code-switching (CS), a ubiquitous phenomenon due to the ease of communication it offers in multilingual communities still remains an understudied problem in language processing. The primary reasons behind this are: (1) minimal efforts in leveraging large pretrained multilingual models, and (2) the lack of annotated data. The distinguishing case of low performance of multilingual models in CS is the intra-sentence mixing of languages leading to switch points. We first benchmark two sequence labeling tasks -POS and NER on 4 different language pairs with a suite of pretrained models to identify the problems and select the best performing model, char-BERT, among them (addressing (1)). We then propose a self training method to repurpose the existing pretrained models using a switch-point bias by leveraging unannotated data (addressing (2)). We finally demonstrate that our approach performs well on both tasks by reducing the gap between the switch point performance while retaining the overall performance on two distinct language pairs in both the tasks 1 .

show abstract

Dependency Parsing of Code-Switching Data with Cross-Lingual Feature Representations

Cited by 11 publications

References 16 publications

A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies

A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies

Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching

Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching

Contact Info

Product

Resources

About