A Deep Generative Model for Code Switched Text

Samanta, Bidisha; Reddy, Sharmila; Jagirdar, Hussain; Ganguly, Niloy; Chakrabarti, Soumen

doi:10.24963/ijcai.2019/719

Cited by 25 publications

(23 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their method needs an external NMT system to obtain monolingual fragment from code-switched text and is expensive to scale to more language pairs. Garg et al (2018) Samanta et al (2019) propose a two-level hierarchical variational autoencoder that models syntactic signals in the lower layer and language switching signals in the upper layer. Their model can leverage modest real code-switched text and large monolingual text to generate large amounts of codeswitched text along with its language at token level.…”

Section: Code-mixed Data Generationmentioning

confidence: 99%

Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing

Jawahar¹,

Nagoudi²,

Abdul-Mageed³

et al. 2021

Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching

View full text Add to dashboard Cite

We describe models focused at the understudied problem of translating between monolingual and code-mixed language pairs. More specifically, we offer a wide range of models that convert monolingual English text into Hinglish (code-mixed Hindi and English). Given the recent success of pretrained language models, we also test the utility of two recent Transformer-based encoder-decoder models (i.e., mT5 and mBART) on the task finding both to work well. Given the paucity of training data for code-mixing, we also propose a dependency-free method for generating code-mixed texts from bilingual distributed representations that we exploit for improving language model performance. In particular, armed with this additional data, we adopt a curriculum learning approach where we first finetune the language models on synthetic data then on gold code-mixed data. We find that, although simple, our synthetic code-mixing method is competitive with (and in some cases is even superior to) several standard methods (backtranslation, method based on equivalence constraint theory) under a diverse set of conditions. Our work shows that the mT5 model, finetuned following the curriculum learning procedure, achieves best translation performance (12.67 BLEU). Our models place first in the overall ranking of the English-Hinglish official shared task.

show abstract

Section: Code-mixed Data Generationmentioning

confidence: 99%

Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing

Jawahar¹,

Nagoudi²,

Abdul-Mageed³

et al. 2021

Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching

View full text Add to dashboard Cite

show abstract

“…Most recently, Winata et al (2019a) utilized the language-agnostic metarepresentation method to represent the code-mixed sentences. There are also other studies (Adel et al, 2013a(Adel et al, ,b, 2015Choudhury et al, 2017;Winata et al, 2018;Gonen and Goldberg, 2018;Samanta et al, 2019) for code-mixed language modelling.…”

Section: Related Workmentioning

confidence: 95%

A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning

Gupta

Ekbal

Bhattacharyya

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Code-mixing, the interleaving of two or more languages within a sentence or discourse is ubiquitous in multilingual societies. The lack of code-mixed training data is one of the major concerns for the development of end-to-end neural network-based models to be deployed for a variety of natural language processing (NLP) applications. A potential solution is to either manually create or crowd-source the code-mixed labelled data for the task at hand, but that requires much human efforts and often not feasible because of the language specific diversity in the code-mixed text. To circumvent the data scarcity issue, we propose an effective deep learning approach for automatically generating the code-mixed text from English to multiple languages without any parallel data. In order to train the neural network, we create synthetic code-mixed texts from the available parallel corpus by modelling various linguistic properties of code-mixing. Our codemixed text generator is built upon the encoderdecoder framework, where the encoder is augmented with the linguistic and task-agnostic features obtained from the transformer based language model. We also transfer the knowledge from a neural machine translation (NMT) to warm-start the training of code-mixed generator. Experimental results and in-depth analysis show the effectiveness of our proposed code-mixed text generation on eight diverse language pairs.

show abstract

“…However, most of them are not directly suitable for generating bilingual code-mixed text, due to the unavailability of sufficient volume of gold-tagged codemixed text. Samanta et al (2019) proposed a generative method using a handful of gold-tagged data; but they cannot produce sentence level tags. Recently, Pratapa et al (2018) used linguistic constraints arising from Equivalence Constraint Theory to design a code-switching grammar that guides text synthesis.…”

Section: Related Workmentioning

confidence: 99%

Improved Sentiment Detection via Label Transfer from Monolingual to Synthetic Code-Switched Text

Samanta¹,

Ganguly²,

Chakrabarti³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Multilingual writers and speakers often alternate between two languages in a single discourse, a practice called "code-switching". Existing sentiment detection methods are usually trained on sentiment-labeled monolingual text. Manually labeled code-switched text, especially involving minority languages, is extremely rare. Consequently, the best monolingual methods perform relatively poorly on code-switched text. We present an effective technique for synthesizing labeled code-switched text from labeled monolingual text, which is more readily available. The idea is to replace carefully selected subtrees of constituency parses of sentences in the resource-rich language with suitable token spans selected from automatic translations to the resource-poor language. By augmenting scarce humanlabeled code-switched text with plentiful synthetic code-switched text, we achieve significant improvements in sentiment labeling accuracy (1.5%, 5.11%, 7.20%) for three different language pairs (English-Hindi, English-Spanish and English-Bengali). We also get significant gains for hate speech detection: 4% improvement using only synthetic text and 6% if augmented with real text.

show abstract

A Deep Generative Model for Code Switched Text

Cited by 25 publications

References 23 publications

Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing

Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing

A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning

Improved Sentiment Detection via Label Transfer from Monolingual to Synthetic Code-Switched Text

Contact Info

Product

Resources

About