Semi-Supervised and Transfer Learning Approaches for Low Resource Sentiment Classification

Abstract

Sentiment classification involves quantifying the affective reaction of a human to a document, media item or an event. Although researchers have investigated several methods to reliably infer sentiment from lexical, speech and body language cues, training a model with a small set of labeled datasets is still a challenge. For instance, in expanding sentiment analysis to new languages and cultures, it may not always be possible to obtain comprehensive labeled datasets. In this paper, we investigate the application of semisupervised and transfer learning methods to improve performances on low resource sentiment classification tasks. We experiment with extracting dense feature representations, pre-training and manifold regularization in enhancing the performance of sentiment classification systems. Our goal is a coherent implementation of these methods and we evaluate the gains achieved by these methods in matched setting involving training and testing on a single corpus setting as well as two cross corpora settings. In both the cases, our experiments demonstrate that the proposed methods can significantly enhance the model performance against a purely supervised approach, particularly in cases involving a handful of training data.

Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning

“…Low-resource NLP Previous work in low-resource NLP tasks includes feature-engineering (Tan and Zhang, 2008) which requires a recurring effort when adapting to a new data set. Another approach is to transfer knowledge across domains to increase the amount of data that is available for training (Zoph et al, 2016;Nguyen and Chiang, 2017;Kocmi and Bojar, 2018;Yang et al, 2017;Gupta et al, 2018). One of these approaches relied on adversarial training (Goodfellow et al, 2014) to learn a domain adaptive classifier (Ganin et al, 2016) in another domain or language where training data was plentiful while ensuring that the model generalizes to the low-resource domain (Chen et al, 2018).…”

“…Adapting pre-trained models The effectiveness of transfer learning in low-resource settings was previously demonstrated for machine translation (Zoph et al, 2016;Nguyen and Chiang, 2017;Kocmi and Bojar, 2018), sequence tagging (Yang et al, 2017) and sentiment classification (Gupta et al, 2018 close to target corpus. Some previous work has analyzed the performance behavior of the BERT model in different scenarios.…”

Affective Concept-Based Encoding of Patient Narratives via Sentic Computing and Neural Networks

“…Semisupervised learning is a fundamental solution that allows learning from heterogeneous unlabeled examples spread via social network in combination with typically sets of labeled data. In this sense, several studies used semisupervised techniques to primarily develop automated corpora [23] by utilizing word representation capabilities in two ways: (1) generating context-depend embeddings, (2) transferring knowledge from pretrained embeddings or fusing weighted distributed features [24][25][26]. The ultimate problem is that these corpora result in low medical entities recognition recall due to the failure of recognizing drugrelated components and their aspects such as drug reactions and indications Extracting drug-related explicit aspects may be failed due to the existence of adverse drug reaction (ADR) multi-word expressions, leading to a higher accumulation of false positives and negatives.…”

Data Augmentation for Low Resource Sentiment Analysis Using Generative Adversarial Networks

“…Survey articles such as ones by Liu et al [12] and Medhat et al [13] provide a summary of such techniques. Sentiment analysis has been previously studied in low resource settings by application of methods such as transfer learning [14] and semi-supervised learning [15]. The set of techniques proposed for sentiment analysis in absence of labeled data include manifold regularization [14], semi-supervised recursive autoencoders [16], document word co-regularization [17] and latent variable models [18].…”

“…Sentiment analysis has been previously studied in low resource settings by application of methods such as transfer learning [14] and semi-supervised learning [15]. The set of techniques proposed for sentiment analysis in absence of labeled data include manifold regularization [14], semi-supervised recursive autoencoders [16], document word co-regularization [17] and latent variable models [18]. On the other hand, GAN models were proposed in 2014 and a tutorial by Goodfellow [19] provides a background on GANs.…”