kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification

Abstract

Code switching is a linguistic phenomenon that may occur within a multilingual setting where speakers share more than one language. With the increasing communication between groups with different languages, this phenomenon is more and more popular. However, there are little research and data in this area, especially in code-mixing sentiment classification. In this work, the domain transfer learning from state-of-the-art uni-language model ERNIE is tested on the code-mixing dataset, and surprisingly, a strong baseline is achieved. Furthermore, the adversarial training with a multi-lingual model is used to achieve 1st place of SemEval-2020 Task 9 Hindi-English sentiment classification competition.

AdvCodeMix: Adversarial Attack on Code-Mixed Data

“…A novel and interesting approach, BERT-Attack, was also proposed [6] where the pre-trained BERT model was used to effectively attack fine-tuned BERT models, as well as traditional LSTM based deep learning models. Liu et al [7] have applied the concept of transfer learning using the ERNIE framework, along with adversarial training using a multilingual model, to their work. Tan et al [20] have proposed two strong black-box adversarial attack frameworks-one word-level and another phrase-level, the latter one being being particularly effective on XNLI.…”