Detecting Entailment in Code-Mixed Hindi-English Conversations

Abstract

The presence of large-scale corpora for Natural Language Inference (NLI) has spurred deep learning research in this area, though much of this research has focused solely on monolingual data. Code-mixing is the intertwined usage of multiple languages, and is commonly seen in informal conversations among polyglots. Given the rising importance of dialogue agents, it is imperative that they understand code-mixing, but the scarcity of code-mixed Natural Language Understanding (NLU) datasets has precluded research in this area. The dataset by Khanuja et al. (2020a) for detecting conversational entailment in codemixed Hindi-English text is the first of its kind. We investigate the effectiveness of language modeling, data augmentation, translation, and architectural approaches to address the codemixed, conversational, and low-resource aspects of this dataset. We obtain +8.09% test set accuracy over the current state of the art.

Bilingual Low-Resource Neural Machine Translation with Round-Tripping: The Case of Persian-Spanish

“…We compare the performance of our systems against the system with the highest test set performance discussed in Chakravarthy et al (2020) and the baselines provided by Khanuja et al (2020b). The performance of our systems is shown in Table 5.…”

“…The confusion matrix for the predictions from our best model is shown Model Accuracy mBERT (Khanuja et al, 2020b) 61.09 Mod. mBERT (Khanuja et al, 2020b) 63.1 mod-mBERT (Chakravarthy et al, 2020) 62.41…”

“…It consists of conversations from Hindi movies (Bollywood) as premises. Chakravarthy et al (2020) compare the effectiveness of various approaches on the dataset.…”

The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding

“…This was extended for a variety of code-switched tasks by Khanuja et al (2020b), where they showed improvements on several tasks using MLM pretraining on real and synthetic code-switched text. Chakravarthy et al (2020) further improved the NLI performance of mBERT by including large amounts of in-domain code-switched text during MLM pretraining. Gururangan et al (2020) empirically demonstrate that pretraining is most beneficial when the domains of the intermediate and target tasks are similar, which we observe as well.…”

Exploring Graph-Algebraic CCG Combinators for Syntactic-Semantic AMR Parsing

“…NLP tools for monolingual and multilingual language processing have rapidly progressed in the past few years; thanks to the transformer-based models such as Multilingual BERT (Devlin et al, 2019) & XLM-RoBERTa (Conneau et al, 2020), and their pretraining techniques. On various mixed datasets, recent studies have shown that adopting multilingual pretrained models can perform better than their previous deep learning counterparts (Pires et al, 2019;Khanuja et al, 2020;Chakravarthy et al, 2020;Jayanthi and Gupta, 2021). While this looks promising for multilingual, the same is not translated to code-mixing.…”