Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space

Abstract

In this paper, we propose a novel data augmentation method, referred to as Controllable Rewriting based Question Data Augmentation (CRQDA), for machine reading comprehension (MRC), question generation, and question-answering natural language inference tasks. We treat the question data augmentation task as a constrained question rewriting problem to generate context-relevant, high-quality, and diverse question data samples. CRQDA utilizes a Transformer autoencoder to map the original discrete question into a continuous embedding space. It then uses a pre-trained MRC model to revise the question representation iteratively with gradientbased optimization. Finally, the revised question representations are mapped back into the discrete space, which serve as additional question data. Comprehensive experiments on SQuAD 2.0, SQuAD 1.1 question generation, and QNLI tasks demonstrate the effectiveness of CRQDA 1 .

Data Augmentation Approaches in Natural Language Processing: A Survey

“…Paraphrasing Thesauruses Zhang et al [5], Wei et al [6], Coulombe et al [7] Semantic Embeddings Wang et al [8] MLMs Jiao et al [9] Rules Coulombe et al [7], Regina et al [10], Louvan et al [11] Machine Translation Back-translation Xie et al [12], Zhang et al [13] Unidirectional Translation Nishikawa et al [14], Bornea et al [15] Model Generation Hou et al [16], Li et al [17], Liu et al [18] Noising Swapping Wei et al [6], Luque et al [19], Yan et al [20] Deletion Wei et al [6], Peng et al [21], Yu et al [22] Insertion Wei et al [6], Peng et al [21], Yan et al [20] Substitution Coulombe et al [7], Xie et al [23], Louvan et al [11] Mixup Guo et al [24], Cheng et al [25] Sampling Rules Min et al [26], Liu et al [27] Seq2Seq Models Kang et al [28], Zhang et al [13], Raille et al [29] Language Models…”

“…9 Kober et al [68] use GAN to generate samples that are very similar to the original data. Liu et al [18] employ a pre-trained model to share the question embeddings and the guidance for the proposed Transformer-based model. Then the proposed model could generate both context-relevant answerable questions and unanswerable questions.…”

Substructure Substitution: Structured Data Augmentation for NLP

“…Label-conditioned text generation. Recent work has explored generating new examples by training a conditional text generation model (Bergmanis et al, 2017;Liu et al, 2020a;Ding et al, 2020;Liu et al, 2020b, inter alia), or applying post-processing on the examples generated by pretrained models Wan et al, 2020;Yoo et al, 2020). In the data augmentation stage, given labels in the original dataset as conditions, such models generate associated text accordingly.…”

Cheap and Good? Simple and Effective Data Augmentation for Low Resource Machine Reading

“…We pose the data augmentation problem [9,11] as a positive example expansion problem. First, we sample which answers should be used for augmentation with probability 𝑝.…”