Automatic Text Summarization for Moroccan Arabic Dialect Using an Artificial Intelligence Approach

Gaanoun, Kamel; Naira, Abdou Mohamed; Allak, Anass; Benelallam, Imade

doi:10.1007/978-3-031-06458-6_13

“…This representation is fed to the encoder with BILSTM to generate the sentence representations of the sentence. Finally, using a pre-trained Named Entity Recognition (NER) model called "Marefa-NLP/marefa-ner" (Gaanoun, et al 2022), we labeled some information from the extractive summary as each item was classi ed into one of the nine categories (person, location, organization, nationality, job, product, event, time, and artwork). This model used an auto tokenizer to tokenize sentences with some hyper-parameters like padding = true, truncation = true, and return tensors = "p" to give weight to sentences and sentence position and to get the relationships among sentences to represent hidden states.…”

Section: Sequence To Sequence Modelmentioning

confidence: 99%

A Hybrid Arabic text summarization Approach based on Seq-to-seq and Transformer

Elsaid

¹

,

Mohamed

²

,

Fattouh

³

et al. 2023

Preprint

0

View full text Add to dashboard Cite

Text summarization is essential in natural language processing as the data volume increases quickly. Therefore, the user needs to summarize that data into a meaningful text in a short time. There are two common methods of text summarization: extractive and abstractive. There are many efforts to summarize Latin texts. However, summarizing Arabic texts is challenging for many reasons, including the language’s complexity, structure, and morphology. Also, there is a need for benchmark data sources and a gold standard Arabic evaluation metrics summary. Thus, the contribution of this paper is multi-fold: First, the paper proposes a hybrid approach consisting of a Modified Sequence-To-Sequence (MSTS) model and a transformer-based model. The seq-to-seq-based model is modified by adding multi-layer encoders and a one-layer decoder to its structure. The output of the MSTS model is the extractive summarization. To generate the abstractive summarization, the extractive summarization is manipulated by a transformer-based model. Second, it introduces a new Arabic benchmark dataset, called the HASD, which includes 43k articles with their extractive and abstractive summaries. Third, this work modifies the well-known extractive EASC benchmarks by adding to each text its abstractive summarization. Finally, this paper proposes a new measure called the Arabic-rouge measure for the abstractive summary depending on structure and similarity between words. The proposed method is tested using the proposed HASD and Modified EASC benchmarks and evaluated using Rouge, Bleu, and Arabic Rouge. The experimental results show satisfactory results compared to state-of-the-art methods.

show abstract

“…This representation is fed to the encoder with BILSTM to generate the sentence representations of the sentence. Finally, using a pre-trained Named Entity Recognition (NER) model called "Marefa-NLP/marefa-ner" (Gaanoun, et al 2022), we labeled some information from the extractive summary as each item was classified into one of the nine categories (person, location, organization, nationality, job, product, event, time, and artwork). This model used an auto tokenizer to tokenize sentences with some hyper-parameters like padding = true, truncation = true, and return tensors = "p" to give weight to sentences and sentence position and to get the relationships among sentences to represent hidden states.…”

Section: Encodermentioning

confidence: 99%

Hybrid Arabic text summarization Approach based on Seq-to-seq and Transformer

Elsaid

¹

,

Mohammed

²

,

Fattouh

³

et al. 2023

Preprint

0

View full text Add to dashboard Cite

Text summarization is essential in natural language processing as the data volume increases quickly. Therefore, the user needs to summarize that data into a meaningful text in a short time. There are two common methods of text summarization: extractive and abstractive. There are many efforts to summarize Latin texts. However, summarizing Arabic texts is challenging for many reasons, including the language’s complexity, structure, and morphology. Also, there is a need for benchmark data sources and a gold standard Arabic evaluation metrics summary. Thus, the contribution of this paper is multi-fold: First, the paper proposes a hybrid approach consisting of a Modified Sequence-To-Sequence (MSTS) model and a transformer-based model. The seq-to-seq-based model is modified by adding multi-layer encoders and a one-layer decoder to its structure. The output of the MSTS model is the extractive summarization. To generate the abstractive summarization, the extractive summarization is manipulated by a transformer-based model. Second, it introduces a new Arabic benchmark dataset, called the HASD, which includes 43k articles with their extractive and abstractive summaries. Third, this work modifies the well-known extractive EASC benchmarks by adding to each text its abstractive summarization. Finally, this paper proposes a new measure called the Arabic-rouge measure for the abstractive summary depending on structure and similarity between words. The proposed method is tested using the proposed HASD and Modified EASC benchmarks and evaluated using Rouge, Bleu, and Arabic Rouge. The experimental results show satisfactory results compared to state-of-the-art methods.

show abstract

“…In Abstractive Summarization, the primary challenge lies in obtaining labeled data, as it necessitates documents along with their summaries for model training. Many existing works employing this method rely on press articles [10,12,28,30]. This approach typically involves treating the articles and their headlines as the summaries.…”

Section: Text Summarizationmentioning

confidence: 99%

“…Our proposed architecture draws inspiration from the works in [10,14,29]. It follows the pipeline outlined in Figure 4 and is structured as follows:…”

Section: Text Summarizationmentioning

confidence: 99%

See 1 more Smart Citation

BantuLM: Enhancing Cross-Lingual Learning in the Bantu Language Family

Mohamed,

Benelallam,

Bahafid

et al. 2023

Preprint

Self Cite

0

View full text Add to dashboard Cite

This paper outlines methods for improving Bantu languages through the application of Natural Language Processing techniques. We trained a Large Language Model known as Bidirectional Encoder Representations from Transformers for the understanding of 18 Bantu languages. More precisely, we pre-trained the model using an unsupervised corpus obtained using pseudo-labeling. This pre-training task aims to comprehend the latent structures of these languages owing to an attention mechanism that enables a deeper understanding of the context. We then conducted various experiments on five downstream tasks: Language Identification, Sentiment Analysis, News Classification, Named Entity Recognition and Text Summarization. Finally, we proposed to test the effectiveness of using multilingualism in a few closely related languages instead of leveraging a vast amount of data and multiple languages that are not necessarily related. In fact, we conducted experiments on unseen languages belonging to the Bantu family and we found that the model demonstrates better ability understanding them due to their similarities to the languages used for pre-training.

show abstract

Automatic Text Summarization for Moroccan Arabic Dialect Using an Artificial Intelligence Approach

Cited by 8 publications

References 20 publications

A Hybrid Arabic text summarization Approach based on Seq-to-seq and Transformer

A Hybrid Arabic text summarization Approach based on Seq-to-seq and Transformer

Hybrid Arabic text summarization Approach based on Seq-to-seq and Transformer

BantuLM: Enhancing Cross-Lingual Learning in the Bantu Language Family

Contact Info

Product

Resources

About