Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification

Dong, Xin; Zhu, Yaxin; Zhang, Yupeng; Fu, Zuohui; Xu, Daolin; Yang, Sen; Melo, Gerard de

doi:10.1145/3397271.3401209

Cited by 21 publications

(13 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Later, realizing that the common one-hot representation of text does not admit perturbations, Miyato et al [20] instead define the adversarial perturbations on continuous word embeddings to successfully extend AT to text classification. Dong et al [7] further propose a self-learning based method to employ AT to cross-lingual text classification. In addition, some methods have also been proposed to explore the use of AT to model other tasks.…”

Section: Adversarial Training (At)mentioning

confidence: 99%

Enhancing Knowledge Tracing via Adversarial Training

Guo

Huang

Gao

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

We study the problem of knowledge tracing (KT) where the goal is to trace the students' knowledge mastery over time so as to make predictions on their future performance. Owing to the good representation capacity of deep neural networks (DNNs), recent advances on KT have increasingly concentrated on exploring DNNs to improve the performance of KT. However, we empirically reveal that the DNNs based KT models may run the risk of overfitting, especially on small datasets, leading to limited generalization. In this paper, by leveraging the current advances in adversarial training (AT), we propose an efficient AT based KT method (ATKT) to enhance KT model's generalization and thus push the limit of KT. Specifically, we first construct adversarial perturbations and add them on the original interaction embeddings as adversarial examples. The original and adversarial examples are further used to jointly train the KT model, forcing it is not only to be robust to the adversarial examples, but also to enhance the generalization over the original ones. To better implement AT, we then present an efficient attentive-LSTM model as KT backbone, where the key is a proposed knowledge hidden state attention module that adaptively aggregates information from previous knowledge hidden states while simultaneously highlighting the importance of current knowledge hidden state to make a more accurate prediction. Extensive experiments on four public benchmark datasets demonstrate that our ATKT achieves new state-of-the-art performance. Code is available at: https://github.com/xiaopengguo/ATKT. CCS CONCEPTS• Applied computing → Computer-assisted instruction; Learning management systems.

show abstract

Section: Adversarial Training (At)mentioning

confidence: 99%

Enhancing Knowledge Tracing via Adversarial Training

Guo

Huang

Gao

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…Dai et al [19] put forward a novel metric-based GAN, which used the distance-criteria to distinguish between real and fake samples. Dong et al [20] presented a semi-supervised adversarial training process for cross-lingual text classification, where the labeled data from one language could be applied to a completely different language classification. We also refer to various solutions for imbalance datasets.…”

Section: Related Work 21 Adversarial Neural Networkmentioning

confidence: 99%

Machine Translation in Low-Resource Languages by an Adversarial Neural Network

et al. 2021

View full text Add to dashboard Cite

Existing Sequence-to-Sequence (Seq2Seq) Neural Machine Translation (NMT) shows strong capability with High-Resource Languages (HRLs). However, this approach poses serious challenges when processing Low-Resource Languages (LRLs), because the model expression is limited by the training scale of parallel sentence pairs. This study utilizes adversary and transfer learning techniques to mitigate the lack of sentence pairs in LRL corpora. We propose a new Low resource, Adversarial, Cross-lingual (LAC) model for NMT. In terms of the adversary technique, LAC model consists of a generator and discriminator. The generator is a Seq2Seq model that produces the translations from source to target languages, while the discriminator measures the gap between machine and human translations. In addition, we introduce transfer learning on LAC model to help capture the features in rare resources because some languages share the same subject-verb-object grammatical structure. Rather than using the entire pretrained LAC model, we separately utilize the pretrained generator and discriminator. The pretrained discriminator exhibited better performance in all experiments. Experimental results demonstrate that the LAC model achieves higher Bilingual Evaluation Understudy (BLEU) scores and has good potential to augment LRL translations.

show abstract

“…Lai et al (2019) proposed using an unlabeled corpus in the target language to bridge the gap between the language and the domain. Dong et al (2020) 2019) concatenated contextualized representations with knowledge graph embeddings to represent author entities and used them as features for the book classification task. E-BERT (Poerner et al, 2020) inserts KB entities next to the entity names in the input sequence to improve BERT's performance for entity-centric tasks.…”

Section: Improving Cross-lingual Transfer Learningmentioning

confidence: 99%

A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification

Nishikawa¹,

Yamada²,

Tsuruoka³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a multilingual bag-of-entities model that effectively boosts the performance of zero-shot cross-lingual text classification by extending a multilingual pre-trained language model (e.g., M-BERT). It leverages the multilingual nature of Wikidata: entities in multiple languages representing the same concept are defined with a unique identifier. This enables entities described in multiple languages to be represented using shared embeddings. A model trained on entity features in a resourcerich language can thus be directly applied to other languages. Our experimental results on cross-lingual topic classification (using the MLDoc and TED-CLDC datasets) and entity typing (using the SHINRA2020-ML dataset) show that the proposed model consistently outperforms state-of-the-art models.

show abstract

Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification

Cited by 21 publications

References 27 publications

Enhancing Knowledge Tracing via Adversarial Training

Enhancing Knowledge Tracing via Adversarial Training

Machine Translation in Low-Resource Languages by an Adversarial Neural Network

A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification

Contact Info

Product

Resources

About