Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Abstract

Recurrent Neural Networks (RNNs) and transformers are deep learning models that have achieved remarkable success in several Natural Language Processing (NLP) tasks since they do not rely on handcrafted features nor enormous knowledge resources. Named Entity Recognition (NER) is an essential NLP task that is used in many applications such as information retrieval, question answering, and machine translation. NER aims to locate, extract, and classify named entities into predefined categories such as person, organization and location. Arabic NER is considered a challenging task because of the complexity and the unique characteristics of Arabic. Most of the previous research on deep learning based-Arabic NER focused on Modern Standard Arabic and Dialectal Arabic, which are different variations from Classical Arabic. In this paper, we investigate deep learning-based Classical Arabic NER using different deep neural network architectures and a BERT based contextual language model that is trained on general domain Arabic text. We propose two RNN-based models by fine-tunning the pretrained BERT language model to recognize and classify named entities from Classical Arabic. The pre-trained BERT contextual language model representations were used as input features to a BGRU/BLSTM model and were fine-tuned using a Classical Arabic NER dataset. In addition, we explore variant architectures of the proposed BERT-BGRU/BLSTM-CRF models. Experimentations showed that the BERT-BGRU-CRF model outperformed the other models by achieving an F-measure of 94.76% on the CANERCorpus. To the best of our knowledge, this is the first work that aims to recognize named entities in Classical Arabic using deep learning.

Dynamic Influence of Network Public Opinions on Price Fluctuation of Small Agricultural Products Based on NLP-TVP-VAR Model—Taking Garlic as an Example

“…Go and Bhayani [38] proposed using a supervised learning method to classify sentiment reviews as positive or negative when extracted from Twitter using naive Bayes, maximum entropy and SVM analysis algorithms, and the experiment showed 80% analysis accuracy. Deep-learning-based sentiment analysis methods are able to actively learn text features and learn semantic information in text based on deep network models to achieve sentiment classification [39]. Dashtipour et al [40] automatically mined relevant features in text information for sentiment classification by convolutional neural network and LSTM model.…”

A Survey on Arabic Named Entity Recognition: Past, Recent Advances, and Future Trends

“…Based on the above proposed Arabic PLMs, Norah et al [71] propose the first work that tackles the classical Arabic NER. They propose to fine-tune a BiLSTM/BiGRU-CRF model on Arabic BERT and compare with two simple baselines, i.e., 1) a linear layer with softmax function on top of BERT (BERT) and 2) a CRF layer on top of BERT (BERT-CRF).…”

Nested Named Entity Recognition Based on Dual Stream Feature Complementation

“…The model alleviates the noise problem in advertising text. Alsaaran et al [ 29 ] proposed the BGRU-CRF named entity recognition model based on BERT. The experiment shows that it is superior to other classical models and has achieved good results on the classic Arabic NER dataset.…”