Mitigating Class Imbalance in Sentiment Analysis through GPT-3-Generated Synthetic Sentences

Abstract

In this paper, we explore the effectiveness of the GPT-3 model in tackling imbalanced sentiment analysis, focusing on the Coursera online course review dataset that exhibits high imbalance. Training on such skewed datasets often results in a bias towards the majority class, undermining the classification performance for minority sentiments, thereby accentuating the necessity for a balanced dataset. Two primary initiatives were undertaken: (1) synthetic review generation via fine-tuning of the Davinci base model from GPT-3 and (2) sentiment classification utilizing nine models on both imbalanced and balanced datasets. The results indicate that good-quality synthetic reviews substantially enhance sentiment classification performance. Every model demonstrated an improvement in accuracy, with an average increase of approximately 12.76% on the balanced dataset. Among all the models, the Multinomial Naïve Bayes achieved the highest accuracy, registering 75.12% on the balanced dataset. This study underscores the potential of the GPT-3 model as a feasible solution for addressing data imbalance in sentiment analysis and offers significant insights for future research.

Enhancing Imbalanced Sentiment Analysis: A GPT-3-Based Sentence-by-Sentence Generation Approach

“…This study is a continuation of our previous work [11], which addressed imbalanced sentiment analysis by fine-tuning the GPT-3 model to generate synthetic data. In addition to determining whether balancing with synthetic data through sentence-by-sentence generation can improve the classification performance, a comparison of the classification performance with that of fine-tuning-based text generation was also conducted.…”

“…In addition to determining whether balancing with synthetic data through sentence-by-sentence generation can improve the classification performance, a comparison of the classification performance with that of fine-tuning-based text generation was also conducted. From the aspect of classification models, the previous study utilized nine traditional machine-learning and deep-learning models [11]. In this study, we focus exclusively on five deep-learning models.…”

Generative Pre-Trained Transformer (GPT) in Research: A Systematic Review on Data Augmentation

“…Quteineh et al [64] present a method combining GPT-2 with Monte Carlo Tree Search for textual data augmentation, significantly boosting classifier performance in active learning with small datasets. Suhaeni et al [65] explore using GPT-3 for generating synthetic reviews to address class imbalances in sentiment analysis, specifically for Coursera course reviews. It shows how synthetic data can enhance the balance and quality of training datasets, leading to improved sentiment classification model performance.…”

“…Literature in [31,35,51,52,56,57] could be attributed to this sub-category. • Text Data Expansion and Enhancement: This involves leveraging GPT to create new textual content and enhance existing datasets, thereby improving machine learning models' performance and addressing data scarcity [12,26,37,39,41,48,53,54,65].…”

Revealing People’s Sentiment in Natural Italian Language Sentences

“…These inherently data-driven learning approaches need an extensive curated dataset and long training, and results are not always accurate and can be affected by bias due to potentially unbalanced training data [14,31]. As no single tool has been found to be sufficiently reliable on its own, some SA solutions use an ensemble approach by combining predictions from multiple models into hybrid tools to improve performance and achieve better accuracy [16,[32][33][34][35].…”