Arabic Named Entity Recognition: A BERT-BGRU Approach

Alsaaran, Norah; Alrabiah, Maha

doi:10.32604/cmc.2021.016054

Cited by 23 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our previous work [16], we used BGRU over fine-tuned BERT model for MSA NER and the results were promising. In this work, we fine-tune a pre-trained BERT language model for the task of Classical Arabic NER using deep learning.…”

Section: Introductionmentioning

confidence: 94%

See 1 more Smart Citation

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Alsaaran

Alrabiah

2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) and transformers are deep learning models that have achieved remarkable success in several Natural Language Processing (NLP) tasks since they do not rely on handcrafted features nor enormous knowledge resources. Named Entity Recognition (NER) is an essential NLP task that is used in many applications such as information retrieval, question answering, and machine translation. NER aims to locate, extract, and classify named entities into predefined categories such as person, organization and location. Arabic NER is considered a challenging task because of the complexity and the unique characteristics of Arabic. Most of the previous research on deep learning based-Arabic NER focused on Modern Standard Arabic and Dialectal Arabic, which are different variations from Classical Arabic. In this paper, we investigate deep learning-based Classical Arabic NER using different deep neural network architectures and a BERT based contextual language model that is trained on general domain Arabic text. We propose two RNN-based models by fine-tunning the pretrained BERT language model to recognize and classify named entities from Classical Arabic. The pre-trained BERT contextual language model representations were used as input features to a BGRU/BLSTM model and were fine-tuned using a Classical Arabic NER dataset. In addition, we explore variant architectures of the proposed BERT-BGRU/BLSTM-CRF models. Experimentations showed that the BERT-BGRU-CRF model outperformed the other models by achieving an F-measure of 94.76% on the CANERCorpus. To the best of our knowledge, this is the first work that aims to recognize named entities in Classical Arabic using deep learning.

show abstract

Section: Introductionmentioning

confidence: 94%

“…During training, the log-probability (15) of the correct tag sequence should be maximized. Then the output sequence with the maximum score is predicted by (16).…”

Section: Tag Prediction Layer: Crfmentioning

confidence: 99%

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Alsaaran

Alrabiah

2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the original FLAT algorithm, the final relative position matrix is obtained by calculating four different relative position distances, then the self-attention matrix is calculated, and the class label corresponding to each position is obtained by outputting the Transformer structure to the CRF network [14][15][16]. The original FLAT only considers the different calculation methods of relative positions, and there is no obvious feature of highlighting different types of relationships.…”

Section: Flat Model With Explicit Relative Position Encodingmentioning

confidence: 99%

A Federated Named Entity Recognition Model with Explicit Relation for Power Grid

Luo¹,

Yao²,

Zhao³

et al. 2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

The power grid operation process is complex, and many operation process data involve national security, business secrets, and user privacy. Meanwhile, labeled datasets may exist in many different operation platforms, but they cannot be directly shared since power grid data is highly privacysensitive. How to use these multi-source heterogeneous data as much as possible to build a power grid knowledge map under the premise of protecting privacy security has become an urgent problem in developing smart grid. Therefore, this paper proposes federated learning named entity recognition method for the power grid field, aiming to solve the problem of building a named entity recognition model covering the entire power grid process training by data with different security requirements. We decompose the named entity recognition (NER) model FLAT (Chinese NER Using Flat-Lattice Transformer) in each platform into a global part and a local part. The local part is used to capture the characteristics of the local data in each platform and is updated using locally labeled data. The global part is learned across different operation platforms to capture the shared NER knowledge. Its local gradients from different platforms are aggregated to update the global model, which is further delivered to each platform to update their global part. Experiments on two publicly available Chinese datasets and one power grid dataset validate the effectiveness of our method.

show abstract

“…Recently, the GRUs and especially the BGRUs, have succeeded in solving several problems, such as, recognition of people from electrocardiogram (ECG) signals [9] and Arabic named entity recognition [10]. BGRU are also used in the field of text recognition, such as, Russian handwritten text recognition [11], but in handwritten Arabic text with large vocabulary, not yet done.…”

Section: Figure 1: Different Shapes Of Arabic Printed Charactersmentioning

confidence: 99%

Improving CNN-BGRU Hybrid Network for Arabic Handwritten Text Recognition

Haboubi¹,

Guesmi²,

Alshammari³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Handwriting recognition is a challenge that interests many researchers around the world. As an exception, handwritten Arabic script has many objectives that remain to be overcome, given its complex form, their number of forms which exceeds 100 and its cursive nature. Over the past few years, good results have been obtained, but with a high cost of memory and execution time. In this paper we propose to improve the capacity of bidirectional gated recurrent unit (BGRU) to recognize Arabic text. The advantages of using BGRUs is the execution time compared to other methods that can have a high success rate but expensive in terms of time and memory. To test the recognition capacity of BGRU, the proposed architecture is composed by 6 convolutional neural network (CNN) blocks for feature extraction and 1 BGRU + 2 dense layers for learning and test. The experiment is carried out on the entire database of institut für nachrichtentechnik/ecole nationale d'ingénieurs de Tunis (IFN/ENIT) without any preprocessing or data selection. The obtained results show the ability of BGRUs to recognize handwritten Arabic script.

show abstract

Arabic Named Entity Recognition: A BERT-BGRU Approach

Cited by 23 publications

References 26 publications

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

A Federated Named Entity Recognition Model with Explicit Relation for Power Grid

Improving CNN-BGRU Hybrid Network for Arabic Handwritten Text Recognition

Contact Info

Product

Resources

About