AraBERT transformer model  for Arabic comments and reviews analysis

Moubtahij, Hicham El; Abdelali, Hajar; Tazi, El Bachir

doi:10.11591/ijai.v11.i1.pp379-387

Cited by 8 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…12 transformer-based encoders are there in a BERT base model. The output of this model will be a vector of size 768, which can be given to a classification layer for the task of classification [25]- [31].…”

Section: Bertmentioning

confidence: 99%

Sentiment analysis of Malayalam tweets using bidirectional encoder representations from transformers: a study

Ramamirtham²

2023

IJEECS

View full text Add to dashboard Cite

Sentiment analysis on views and opinions expressed in Indian regional languages has become the current focus of research. But, compared to a globally accepted language like English, research on sentiment analysis in Indian regional languages like Malayalam are very low. One of the major hindrances is the lack of publicly available Malayalam datasets. This work focuses on building a Malayalam dataset for facilitating sentiment analysis on Malayalam texts and studying the efficiency of a pre-trained deep learning model in analyzing the sentiments latent in Malayalam texts. In this work, a Malayalam dataset has been created by extracting 2,000 tweets from Twitter. The bidirectional encoder representations from transformers (BERT) is a pretrained model that has been used for various natural language processing tasks. This work employs a transformer-based BERT model for Malayalam sentiment analysis. The efficacy of BERT in analyzing the sentiments latent in Malayalam texts has been studied by comparing the performance of BERT with various machine learning models as well as deep learning models. By analyzing the results, it is found that a substantial increase in accuracy of 5% for BERT when compared with that of Bi-GRU, which is the next bestperforming model.

show abstract

Section: Bertmentioning

confidence: 99%

Sentiment analysis of Malayalam tweets using bidirectional encoder representations from transformers: a study

Ramamirtham²

2023

IJEECS

View full text Add to dashboard Cite

show abstract

“…Arabic is a widely spoken and written language with a significant presence in the online world. Researchers in the Arabic world have started to focus on creating resources and language models for the Arabic language; examples of Arabic language models include AraBERT [11], ARBERT, MARBET [12], and CAMeLBERT [13], all of which focus on Modern Standard Arabic (MSA). In addition, there are models that cover Arabic dialects for specific countries.…”

Section: Natural Language Processing (Nlp) Of Kuwaiti Dialectmentioning

confidence: 99%

Q8VaxStance: Dataset Labeling System for Stance Detection towards Vaccines in Kuwaiti Dialect

Alostad,

Dawiek,

Davulcu

2023

BDCC

View full text Add to dashboard Cite

The Kuwaiti dialect is a particular dialect of Arabic spoken in Kuwait; it differs significantly from standard Arabic and the dialects of neighboring countries in the same region. Few research papers with a focus on the Kuwaiti dialect have been published in the field of NLP. In this study, we created Kuwaiti dialect language resources using Q8VaxStance, a vaccine stance labeling system for a large dataset of tweets. This dataset fills this gap and provides a valuable resource for researchers studying vaccine hesitancy in Kuwait. Furthermore, it contributes to the Arabic natural language processing field by providing a dataset for developing and evaluating machine learning models for stance detection in the Kuwaiti dialect. The proposed vaccine stance labeling system combines the benefits of weak supervised learning and zero-shot learning; for this purpose, we implemented 52 experiments on 42,815 unlabeled tweets extracted between December 2020 and July 2022. The results of the experiments show that using keyword detection in conjunction with zero-shot model labeling functions is significantly better than using only keyword detection labeling functions or just zero-shot model labeling functions. Furthermore, for the total number of generated labels, the difference between using the Arabic language in both the labels and prompt or a mix of Arabic labels and an English prompt is statistically significant, indicating that it generates more labels than when using English in both the labels and prompt. The best accuracy achieved in our experiments in terms of the Macro-F1 values was found when using keyword and hashtag detection labeling functions in conjunction with zero-shot model labeling functions, specifically in experiments KHZSLF-EE4 and KHZSLF-EA1, with values of 0.83 and 0.83, respectively. Experiment KHZSLF-EE4 was able to label 42,270 tweets, while experiment KHZSLF-EA1 was able to label 42,764 tweets. Finally, the average value of annotation agreement between the generated labels and human labels ranges between 0.61 and 0.64, which is considered a good level of agreement.

show abstract

“…Several recent studies [12], [13] have trained the bidirectional encoder representations from transformers (BERT) model on Wikipedia and Oscar datasets for the Arabic language. In addition to that, several recent studies [14], [15] have fine-tuned the Arabic BERT model [13] for downstream task SA. the drawback identified from the analysis of existing literature are: i) models not tested on different datasets; ii) some models ignore the context meaning of the sentence; iii) the model using context like BERT fined-tuned using general pre-trained models that affect models performance; and iv) there is room for improvement for reported prediction accuracy.…”

Section: Introductionmentioning

confidence: 99%

ArSentBERT: fine-tuned bidirectional encoder representations from transformers model for Arabic sentiment classification

2023

View full text Add to dashboard Cite

Sentiment analysis in the Arabic language is challenging because of its linguistic complexity. Arabic is complex in words, paragraphs, and sentence structure. Moreover, most Arabic documents contain multiple dialects, writing alphabets, and styles (e.g., Franco-Arab). Nevertheless, fine-tuned bidirectional encoder representations from transformers (BERT) models can provide a reasonable prediction accuracy for Arabic sentiment classification tasks. This paper presents a fine-tuning approach for BERT models for classifying Arabic sentiments. It uses Arabic BERT pre-trained models and tokenizers and includes three stages. The first stage is text preprocessing and data cleaning. The second stage uses transfer-learning of the pre-trained models’ weights and trains all encoder layers. The third stage uses a fully connected layer and a drop-out layer for classification. We tested our fine-tuned models on five different datasets that contain reviews in Arabic with different dialects and compared the results to 11 state-of-the-art models. The experiment results show that our models provide better prediction accuracy than our competitors. We show that the choice of the pre-trained BERT model and the tokenizer type improves the accuracy of Arabic sentiment classification.

show abstract

AraBERT transformer model for Arabic comments and reviews analysis

Cited by 8 publications

References 18 publications

Sentiment analysis of Malayalam tweets using bidirectional encoder representations from transformers: a study

Sentiment analysis of Malayalam tweets using bidirectional encoder representations from transformers: a study

Q8VaxStance: Dataset Labeling System for Stance Detection towards Vaccines in Kuwaiti Dialect

ArSentBERT: fine-tuned bidirectional encoder representations from transformers model for Arabic sentiment classification

Contact Info

Product

Resources

About