Transformer based Contextual Model for Sentiment Analysis of Customer Reviews: A Fine-tuned BERT

Durairaj, Ashok Kumar; Chinnalagu, Anandan

doi:10.14569/ijacsa.2021.0121153

Cited by 20 publications

(12 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BERT pioneered bidirectional training, considering both left and right contexts in all layers [52], [53]. In refining this approach, RoBERTa removed the next sentence prediction objective and integrated dynamic masking during training [54], [55].…”

Section: E Bert Roberta and Albertmentioning

confidence: 99%

Beyond BERT: Exploring the Efficacy of RoBERTa and ALBERT in Supervised Multiclass Text Classification

Sy,

Maceda,

Canon

et al. 2024

IJACSA

View full text Add to dashboard Cite

This study investigates the performance of transformer-based machine learning models, specifically BERT, RoBERTa, and ALBERT, in multiclass text classification within the context of the Universal Access to Quality Tertiary Education (UAQTE) program. The aim is to systematically categorize and analyze qualitative responses to uncover domain-specific patterns in students' experiences. Through rigorous evaluation of various hyperparameter configurations, consistent enhancements in model performance are observed with smaller batch sizes and increased epochs, while optimal learning rates further boost accuracy. However, achieving an optimal balance between sequence length and model efficacy presents nuanced challenges, with instances of overfitting emerging after a certain number of epochs. Notably, the findings underscore the effectiveness of the UAQTE program in addressing student needs, particularly evident in categories such as "Family Support" and "Financial Support," with RoBERTa emerging as a standout choice due to its stable performance during training. Future research should focus on fine-tuning hyperparameter values and adopting continuous monitoring mechanisms to reduce overfitting. Furthermore, ongoing review and modification of educational efforts, informed by evidence-based decision-making and stakeholder feedback, is critical to fulfill students' changing needs effectively.

show abstract

Section: E Bert Roberta and Albertmentioning

confidence: 99%

Beyond BERT: Exploring the Efficacy of RoBERTa and ALBERT in Supervised Multiclass Text Classification

Sy,

Maceda,

Canon

et al. 2024

IJACSA

View full text Add to dashboard Cite

show abstract

“…The highest accuracy of 70% was achieved by XLNet. Durairaj and Chinnalagu [14] suggested a fine-tuned BERT model to predict customer sentiment by using customer reviews from various datasets. The proposed model's performance was compared with SVM, FastText, BiLSTM and hybrid FastText-BiLSTM models.…”

Section: Literature Reviewmentioning

confidence: 99%

Transfer Learning for Sentiment Classification Using Bidirectional Encoder Representations from Transformers (BERT) Model

Areshey

Mathkour

2023

Sensors

View full text Add to dashboard Cite

Sentiment is currently one of the most emerging areas of research due to the large amount of web content coming from social networking websites. Sentiment analysis is a crucial process for recommending systems for most people. Generally, the purpose of sentiment analysis is to determine an author’s attitude toward a subject or the overall tone of a document. There is a huge collection of studies that make an effort to predict how useful online reviews will be and have produced conflicting results on the efficacy of different methodologies. Furthermore, many of the current solutions employ manual feature generation and conventional shallow learning methods, which restrict generalization. As a result, the goal of this research is to develop a general approach using transfer learning by applying the “BERT (Bidirectional Encoder Representations from Transformers)”-based model. The efficiency of BERT classification is then evaluated by comparing it with similar machine learning techniques. In the experimental evaluation, the proposed model demonstrated superior performance in terms of outstanding prediction and high accuracy compared to earlier research. Comparative tests conducted on positive and negative Yelp reviews reveal that fine-tuned BERT classification performs better than other approaches. In addition, it is observed that BERT classifiers using batch size and sequence length significantly affect classification performance.

show abstract

“…In order to test the accuracy of our models (BioBERT and BioGPT) we decided to extend these models by adding a softmax function to determine the relevancy of the searched PubMed articles to the case description. The softmax function report kind of sentiment the searched This approach can be used in a teaching a learning clinical setting but in practice seeking evidence to prove an option PubMed article compared to the given case description [43]. Figure 5 illustrate our Softmax Sentiment Model.…”

Section: Enhancing the Bootstrapping Of Qandamentioning

confidence: 99%

Empowering Transformers for Evidence-Based Medicine

Mohammed,

Fiaidhi,

Shaik

2023

Preprint

View full text Add to dashboard Cite

Breaking the barrier for practicing evidence-based medicine rely on effective methods for rapidly identifying relevant evidences from the body of biomedical literature. An important challenge confronted by the medical practitioners is the long time needed to browse, filter, summarize and compile information from different medical resources. Deep learning can help in solving this based on the automatic question answering (Q&A) and transformers. However, Q&A and transformers technologies are not trained to answer clinical queries that can be used for evidence-based practice nor it can respond to structured clinical questioning protocol like PICO (Patient/Problem, Intervention, Comparison and Outcome). This article describes the use of deep learning techniques for Q&A that is based on transformer models like BERT and GPT to answer PICO clinical questions that can be used for evidence-based practice extracted from sound medical research resources like PubMed. We are reporting acceptable clinical answers that are supported by findings from PubMed. Our transformer methods are reaching an acceptable state of the art performance based on two staged bootstrapping process involving filtering relevant articles followed by identifying articles that support the requested outcome expressed by the PICO question. Moreover, we are also reporting experimentations to empower our bootstrapping techniques with patch attentions to the most important keywords in the clinical case and the PICO questions. Our bootstrapped patched with attention is showing relevancy of the evidences collected based on an entropy metrics.

show abstract

Transformer based Contextual Model for Sentiment Analysis of Customer Reviews: A Fine-tuned BERT

Cited by 20 publications

References 4 publications

Beyond BERT: Exploring the Efficacy of RoBERTa and ALBERT in Supervised Multiclass Text Classification

Beyond BERT: Exploring the Efficacy of RoBERTa and ALBERT in Supervised Multiclass Text Classification

Transfer Learning for Sentiment Classification Using Bidirectional Encoder Representations from Transformers (BERT) Model

Empowering Transformers for Evidence-Based Medicine

Contact Info

Product

Resources

About