An Ensemble of Arabic Transformer-based Models for Arabic Sentiment Analysis

Karfi, Ikram El; Fkihi, Sanaa El

doi:10.14569/ijacsa.2022.0130865

Cited by 6 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [11], They have put into practice an ensemble model based on the AraBERT and CAMe LBERT transformer language models. The balanced dataset, which is made up of reviews of contemporary standard Arabic books, was used to evaluate the suggested ensemble model.…”

Section: Related Workmentioning

confidence: 99%

Evaluation of Hotel Performance with Sentiment Analysis by Deep Learning Techniques

Hameed

Abed²,

Sadiq

2023

Int. J. Interact. Mob. Technol.

View full text Add to dashboard Cite

The subject of sentiment analysis through social media sites has witnessed significant development due to the increasing reliance of people on social media in advertising and marketing, especially after the Corona pandemic. There is no doubt that the prevalence of the Arabic language makes it considered one of the most important languages all over the world. Through human comments, it can know things if they are positive or negative. But in fact, the comments are many, and it takes work to evaluate the place or the product through a detailed reading of each comment. Therefore, this study applied deep learning approaches to this issue to provide final results that could be utilized to differentiate between the comments in the dataset. Arabic Sentiment Analysis was used and gave a percentage for each positive and negative commentary. This work used eight methods of deep learning techniques after using Fast Text as embedding, except Ara BERT. These techniques are the transformer (AraBERT), RNN (Long short-term memory (LSTM), Bidirectional long-short term memory (BI-LSTM), Gated recurrent units (GRUs), Bidirectional Gated recurrent units (BI-GRU)), CNN (like ALEXNET, proposed CNN), and ensemble model (CNN with BI-GRU). The Hotel Arabic Reviews Dataset was utilized to test the models. This paper obtained the following results. In the Ara BERT model, the accuracy is 96.442%. In CNN, like the Alex Net model, the accuracy is 93.78%. In the suggested CNN model, the accuracy is 94.43%. In the suggested LSTM model, the accuracy is 95%. In the suggested BI-LSTM model, the accuracy is 95.11%. The accuracy of the suggested GRU model is 95.07%. The accuracy of the suggested BI-GRU model is 95.02%. The accuracy is 94.52% in the Ensemble CNN with BI-GRU model that has been proposed. Consequently, the AraBERT outperformed the other approaches in terms of accuracy. Because the AraBERT has already been trained on some Arabic Wikipedia entries. The LSTM, BI-LSTM, GRU, and BI-GRU, on the other hand, had comparable outcomes.

show abstract

Section: Related Workmentioning

confidence: 99%

Evaluation of Hotel Performance with Sentiment Analysis by Deep Learning Techniques

Hameed

Abed²,

Sadiq

2023

Int. J. Interact. Mob. Technol.

View full text Add to dashboard Cite

show abstract

“…In Ref. [67], an ensemble learning model for Arabic sentiment analysis was proposed which incorporates a bidirectional long short-term memory (BiLSTM) model and a Generative Pre-trained Transformer (GPT) model. Experiments were conducted that involved separate applications of BiLSTM and GPT and a comparison with the ensemble model.…”

mentioning

confidence: 99%

Contemporary Approaches in Evolving Language Models

Oralbekova,

Mamyrbayev,

Othman

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

This article provides a comprehensive survey of contemporary language modeling approaches within the realm of natural language processing (NLP) tasks. This paper conducts an analytical exploration of diverse methodologies employed in the creation of language models. This exploration encompasses the architecture, training processes, and optimization strategies inherent in these models. The detailed discussion covers various models ranging from traditional n-gram and hidden Markov models to state-of-the-art neural network approaches such as BERT, GPT, LLAMA, and Bard. This article delves into different modifications and enhancements applied to both standard and neural network architectures for constructing language models. Special attention is given to addressing challenges specific to agglutinative languages within the context of developing language models for various NLP tasks, particularly for Arabic and Turkish. The research highlights that contemporary transformer-based methods demonstrate results comparable to those achieved by traditional methods employing Hidden Markov Models. These transformer-based approaches boast simpler configurations and exhibit faster performance during both training and analysis. An integral component of the article is the examination of popular and actively evolving libraries and tools essential for constructing language models. Notable tools such as NLTK, TensorFlow, PyTorch, and Gensim are reviewed, with a comparative analysis considering their simplicity and accessibility for implementing diverse language models. The aim is to provide readers with insights into the landscape of contemporary language modeling methodologies and the tools available for their implementation.

show abstract