Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation

Liu, Ruibo; Xu, Guangxuan; Jia, Chenyan; Ma, Weicheng; Wang, Lili; Vosoughi, Soroush

doi:10.18653/v1/2020.emnlp-main.726

Cited by 52 publications

(27 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Different from this work, we leverage a small labeled dataset using a distillation method to augment training data. Liu et al (2020) proposed a text augmentation method using reinforcement learning to guide conditional text generation. For classification of tweets, Sharifirad et al (2018) conducted data augmentation by using a combination of knowledge graphs to add the related concepts to the original tweets.…”

Section: Related Workmentioning

confidence: 99%

Tracking social media during the COVID-19 pandemic: The case study of lockdown in New York State

Lin

Litvak

2022

Expert Systems with Applications

View full text Add to dashboard Cite

Facing the COVID-19 pandemic, governments have implemented a wide range of policies to contain the spread of the virus. During the pandemic, large amounts of COVID-19-related tweets emerge every day. Real-time processing of daily tweets may offer insights for monitoring public opinion about intervention measures implemented. In this work, lockdown policy in New York State has been set as a target of public opinion research. This task includes two stages, stance detection and opinion monitoring. For the stance detection stage, we explored several combinations of different text representations and classification algorithms, finding that the combination of Long Short-Term Memory (LSTM) with Global Vectors for Word Representation (GloVe) outperforms others. Due to the shortage of labeled data, we adopted the data distillation method for the training data augmentation. The augmentation of the training data allows to improve the performance of the model with a very small amount of manually-labeled data. After applying the distillation method, the accuracy of the model has been significantly improved. Utilizing the enhanced model, automatically classified tweets are analyzed over time to monitor the public opinion. By exploring the tweets in New York from January 22nd until September 30th, 2020, we show the correlation of public opinion with COVID-19 cases and mortality data, and the effect of government responses on the opinion shift. These results demonstrate the capability of the presented method to effectively and efficiently monitor public opinion during a pandemic.

show abstract

Section: Related Workmentioning

confidence: 99%

Tracking social media during the COVID-19 pandemic: The case study of lockdown in New York State

Lin

Litvak

2022

Expert Systems with Applications

View full text Add to dashboard Cite

show abstract

“…To make matters worse, even the label for a specific task is changed in some cases after EDA was applied [19]. There have been cases where the sentence became illegible because the word order of the original text was changed [23].…”

Section: A Text Data Augmentationmentioning

confidence: 99%

“…However, the two methods both entail the risk of the aforementioned problems as transformation is applied to the original text data at the token level. EDA has been found to change the semantic elements of the original input text [22], whereas back-translation has been reported to generate inaccurate text depending on the performance of the translator [17] or the vocabulary of the augmented text is reduced as specific tokens are repeated [23].…”

Section: Introductionmentioning

confidence: 99%

Text Embedding Augmentation Based on Retraining With Pseudo-Labeled Adversarial Embedding

Kim

Kang

2022

IEEE Access

View full text Add to dashboard Cite

Pre-trained language models (LMs) have been shown to achieve outstanding performance in various natural language processing tasks; however, these models have a significantly large number of parameters to handle large-scale text corpora during the pre-training process, and thus, they entail the risk of overfitting when fine-tuning for small task-oriented datasets is conducted. In this paper, we propose a text embedding augmentation method to prevent such overfitting. The proposed method applies augmentation to a text embedding by generating an adversarial embedding, which is not identical to original input embedding but maintaining the characteristics of the original input embedding, using PGD-based adversarial training for input text data. A pseudo-label that is identical to the label of the input text is then assigned to adversarial embedding to conduct retraining by using adversarial embedding and pseudo-label as input embedding and label pair for a separate LM. Experimental results on several text classification benchmark datasets demonstrated that the proposed method effectively prevented overfitting, which commonly occurs when adjusting a large-scale pre-trained LM to a specific task.INDEX TERMS Text embedding augmentation, adversarial training, pseudo-label, generating, retraining.

show abstract

“…Augmentation in NLP Data augmentation for NLP has been studied extensively in the past (Jia and Liang, 2016;Silfverberg et al, 2017;Fürstenau and Lapata, 2009). Common methods include those that alter the surface form text (Wei and Zou, 2019) or perturb a latent embedding space (Wang and Yang, 2015;Fadaee et al, 2017;Liu et al, 2020), as well as those that perform paraphrasing (Zhang et al, 2019). Alternatively, masked language models generate new examples by proposing context-aware replacements for the masked token (Kobayashi, 2018;Wu et al, 2019).…”

Section: Data Augmentationmentioning

confidence: 99%

GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation

Chen¹,

Zhou²

2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Practical dialogue systems require robust methods of detecting out-of-scope (OOS) utterances to avoid conversational breakdowns and related failure modes. Directly training a model with labeled OOS examples yields reasonable performance, but obtaining such data is a resource-intensive process. To tackle this limited-data problem, previous methods focus on better modeling the distribution of in-scope (INS) examples.We introduce GOLD as an orthogonal technique that augments existing data to train better OOS detectors operating in low-data regimes. GOLD generates pseudo-labeled candidates using samples from an auxiliary dataset and keeps only the most beneficial candidates for training through a novel filtering mechanism. In experiments across three target benchmarks, the top GOLD model outperforms all existing methods on all key metrics, achieving relative gains of 52.4%, 48.9% and 50.3% against median baseline performance.We also analyze the unique properties of OOS data to identify key factors for optimally applying our proposed method. 1

show abstract

Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation

Cited by 52 publications

References 29 publications

Tracking social media during the COVID-19 pandemic: The case study of lockdown in New York State

Tracking social media during the COVID-19 pandemic: The case study of lockdown in New York State

Text Embedding Augmentation Based on Retraining With Pseudo-Labeled Adversarial Embedding

GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation

Contact Info

Product

Resources

About