Improving Short Text Classification Through Global Augmentation Methods

Marivate, Vukosi; Sefara, Tshephisho Joseph

doi:10.1007/978-3-030-57321-8_21

Cited by 68 publications

(39 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also include data noising techniques, such as altering words in the input of self-encoder networks in order to generate a different sentence (Xie et al 2017;Zolna et al 2017;Li et al 2018), or introducing noise on the word-embedding level. These methods were analyzed in (Marivate and Sefara 2019). Although a viable option when no access to a formal synonym model exists, they require abundant training data.…”

Section: Related Workmentioning

confidence: 99%

Do Not Have Enough Data? Deep Learning to the Rescue!

Anaby-Tavor¹,

Carmeli²,

Goldbraich³

et al. 2020

AAAI

201

View full text Add to dashboard Cite

Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a powerful pre-trained neural network model to artificially synthesize new labeled data for supervised learning. We mainly focus on cases with scarce labeled data. Our method, referred to as language-model-based data augmentation (LAMBADA), involves fine-tuning a state-of-the-art language generator to a specific task through an initial training phase on the existing (usually small) labeled data. Using the fine-tuned model and given a class label, new sentences for the class are generated. Our process then filters these new sentences by using a classifier trained on the original data. In a series of experiments, we show that LAMBADA improves classifiers' performance on a variety of datasets. Moreover, LAMBADA significantly improves upon the state-of-the-art techniques for data augmentation, specifically those applicable to text classification tasks with little data.

show abstract

Section: Related Workmentioning

confidence: 99%

Do Not Have Enough Data? Deep Learning to the Rescue!

Anaby-Tavor¹,

Carmeli²,

Goldbraich³

et al. 2020

AAAI

201

View full text Add to dashboard Cite

show abstract

“…Hence, textual data can be easily augmented by replacing a fraction of the original text with the nearest neighbours of the chosen words. This approach requires either pre-trained word embedding models for the language in question or enough data from the target application to build the embedding model [16]. Thus, this approach does not require access to a dictionary or thesaurus for a language to find synonyms [16].…”

Section: Semantic Similarity Augmentationmentioning

confidence: 99%

“…This approach requires either pre-trained word embedding models for the language in question or enough data from the target application to build the embedding model [16]. Thus, this approach does not require access to a dictionary or thesaurus for a language to find synonyms [16]. This can be advantageous for languages where such resources are more difficult to obtain, but there is enough unsupervised text data to be able to build the embedding models [16].…”

Section: Semantic Similarity Augmentationmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of data augmentation methods for legal document classification

Csányi

Orosz

2021

Acta Tech. Jaurinensis

View full text Add to dashboard Cite

Sorting out the legal documents by their subject matter is an essential and time-consuming task due to the large amount of data. Many machine learning-based text categorization methods exist, which can resolve this problem. However, these algorithms can not perform well if they do not have enough training data for every category. Text augmentation can resolve this problem. Data augmentation is a widely used technique in machine learning applications, especially in computer vision. Textual data has different characteristics than images, so different solutions must be applied when the need for data augmentation arises. However, the type and different characteristics of the textual data or the task itself may reduce the number of methods that could be applied in a certain scenario. This paper focuses on text augmentation methods that could be applied to legal documents when classifying them into specific groups of subject matters.

show abstract

“…Back-translation (Sennrich et al, 2016;Yu et al, 2018) is also a major way for textual DA, which uses machine translation model to translate English sentences into another language (e.g., French), and back into English. Besides, data noising techniques (Xie et al, 2017;Marivate and Sefara, 2019) and paraphrasing (Kumar et al, 2019) are proposed to generate new textual samples. All the methods mentioned above usually generate individual sentences separately.…”

Section: Related Workmentioning

confidence: 99%

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space

Liu

Gong

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

In this paper, we propose a novel data augmentation method, referred to as Controllable Rewriting based Question Data Augmentation (CRQDA), for machine reading comprehension (MRC), question generation, and question-answering natural language inference tasks. We treat the question data augmentation task as a constrained question rewriting problem to generate context-relevant, high-quality, and diverse question data samples. CRQDA utilizes a Transformer autoencoder to map the original discrete question into a continuous embedding space. It then uses a pre-trained MRC model to revise the question representation iteratively with gradientbased optimization. Finally, the revised question representations are mapped back into the discrete space, which serve as additional question data. Comprehensive experiments on SQuAD 2.0, SQuAD 1.1 question generation, and QNLI tasks demonstrate the effectiveness of CRQDA 1 .

show abstract

Improving Short Text Classification Through Global Augmentation Methods

Cited by 68 publications

References 22 publications

Do Not Have Enough Data? Deep Learning to the Rescue!

Do Not Have Enough Data? Deep Learning to the Rescue!

Comparison of data augmentation methods for legal document classification

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space

Contact Info

Product

Resources

About