RoBERTa-LSTM: A Hybrid Model for Sentiment Analysis With Transformer and Recurrent Neural Network

Tan, Kian Long; Lee, Chin Poo; Anbananthen, Kalaiarasi Sonai Muthu; Lim, Kian Ming

doi:10.1109/access.2022.3152828

Cited by 126 publications

(39 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, in addition to the BERT model, we propose more improved models of BERT, which are robustly optimized BERT pre-training approach (RoBERTa) and DistilBERT. RoBERTa [29]- [31] is improved by removing next sentence prediction during training and giving dynamic masking that will keep changing, longer training time will go with larger batch sizes. RoBERTa is proposed to improve the accuracy of the BERT model.…”

Section: Research Methods 21 Related Algorithmsmentioning

confidence: 99%

Apply deep learning to improve the question analysis model in the Vietnamese question answering system

Phuc

Nghiem²,

Minh³

et al. 2023

IJECE

View full text Add to dashboard Cite

Question answering (QA) system nowadays is quite popular for automated answering purposes, the meaning analysis of the question plays an important role, directly affecting the accuracy of the system. In this article, we propose an improvement for question-answering models by adding more specific question analysis steps, including contextual characteristic analysis, pos-tag analysis, and question-type analysis built on deep learning network architecture. Weights of extracted words through question analysis steps are combined with the best matching 25 (BM25) algorithm to find the best relevant paragraph of text and incorporated into the QA model to find the best and least noisy answer. The dataset for the question analysis step consists of 19,339 labeled questions covering a variety of topics. Results of the question analysis model are combined to train the question-answering model on the data set related to the learning regulations of Industrial University of Ho Chi Minh City. It includes 17,405 pairs of questions and answers for the training set and 1,600 pairs for the test set, where the robustly optimized BERT pre-training approach (RoBERTa) model has an F1-score accuracy of 74%. The model has improved significantly. For long and complex questions, the mode has extracted weights and correctly provided answers based on the question’s contents.

show abstract

Section: Research Methods 21 Related Algorithmsmentioning

confidence: 99%

Apply deep learning to improve the question analysis model in the Vietnamese question answering system

Phuc

Nghiem²,

Minh³

et al. 2023

IJECE

View full text Add to dashboard Cite

show abstract

“…The authors of [32] employed topic modelling and sentiment analysis on tweets regarding global climate change to reveal public opinion. The authors of [36] used "IMDB", "Twitter-US-Airline-Sentiment", and "Sentiment140" datasets, and "Roberta-LSTM" outperforms state-of-the-art sentiment analysis algorithms. The paper's contributions are threefold:…”

Section: Related Workmentioning

confidence: 99%

Aspect based Sentiment & Emotion Analysis with ROBERTa, LSTM

Sirisha¹,

Chandana²

2022

IJACSA

View full text Add to dashboard Cite

Internet usage has increased social media over the past few years, significantly impacting public opinion on online social networks. Nowadays, these websites are considered the most appropriate place to express feelings and opinions. The popular social media site Twitter offers valuable insight into people's thoughts. Throughout the conflict between Russia and Ukraine, people from all over the world have expressed their opinions. In this study, "machine-learning" and "deep-learning" techniques are used to understand people's emotions and their views about this war are revealed. This study unveils a novel deeplearning approach that merges the best features of the sequence and transformer models while fixing their respective flaws. The model combines Roberta with ABSA (Aspect based sentiment analysis) and Long Short-Term Memory for sentiment analysis. A large dataset of geographically tagged tweets related to the Ukraine-Russia war was collected from Twitter. We analyzed this dataset using the Roberta-based sentiment model. In contrast, the Long Short-Term Memory model can effectively capture longdistance contextual semantics. The Robustly optimized BERT with ABSA approach maps words into a compact, meaningful word embedding space. The accuracy of the suggested hybrid model is 94.7%, which is higher than the accuracy of the stateof-the-art techniques.

show abstract

“…A hybrid deep learning method is proposed by [21] and has concluded that RoBERTa-LSTM model benefits from the strengths of both RoBERTa and LSTM, where RoBERTa efficiently encodes the words into word embedding while LSTM excels in capturing the long-distance dependencies…”

Section: Related Workmentioning

confidence: 99%

Deciphering Voice of the Customer using Text Analytics and Sentiment Analysis: An Interpretable Review Rating Prediction using RoBERTa

U.C.

N.M.

2022

IJRITCC

View full text Add to dashboard Cite

In this era of cut-throat competition among traditional and newer digital organizations, capturing, listening, and understanding customer voices are critical for success in the marketplace. The challenge to decipher the voice of the customer (VOC) has multiplied many times today, as now the number of customer reviews are present in multiple platforms and the data to be analyzed is huge. Sentiment analysis, and text analytics using machine learning, deep learning tools and transformer-based tools can be applied to gather meaningful insights from these data. This paper applies the traditional machine learning tools of the Naive Bayes classifier, Random Forest and AdaBoost to predict the customer review ratings. These results are compared with deep learning methods of CNN, RNN and Bi-LSTM and transformer-based approaches of BERT, DistilBERT and RoBERTa. The results show that RoBERTa has the highest accuracy among these methods. Paper also uses the explainable AI tool of LIME to understand the customer sentiments deeper in terms of why customers are giving a particular rating to the product. Business organizations will continue to use more and more AI tools to understand the customer feedback and the attempt in this paper is to learn how we can make predictions faster and more accurately.

show abstract

RoBERTa-LSTM: A Hybrid Model for Sentiment Analysis With Transformer and Recurrent Neural Network

Cited by 126 publications

References 23 publications

Apply deep learning to improve the question analysis model in the Vietnamese question answering system

Apply deep learning to improve the question analysis model in the Vietnamese question answering system

Aspect based Sentiment & Emotion Analysis with ROBERTa, LSTM

Deciphering Voice of the Customer using Text Analytics and Sentiment Analysis: An Interpretable Review Rating Prediction using RoBERTa

Contact Info

Product

Resources

About