Developing and performance evaluation of a new Arabic heavy/light stemmer

Zeroual, Imad; Boudchiche, Mohamed; Mazroui, Azzeddine; Lakhouaja, Abdelhak

doi:10.1145/3090354.3090371

Cited by 8 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the second step, the root frequency map is constructed to derive the possible stem by minimizing the frequency map of roots. Zeroual [53] used pattern-matching techniques to derive the stem of the Arabic word. For instance, the words ‫ﮐتﺐ‬ [kataba/to write], ‫ﮐاتﺐ‬ [kaatib/writer], ‫مکتوب‬ [makotuwb/written],…”

Section: Related Workmentioning

confidence: 99%

Building a Multilevel Inflection Handling Stemmer to Improve Search Effectiveness for Urdu Language

Jabbar,

Iqbal,

Alaulamie

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Stemming is an essential step in various Natural Language Processing (NLP) applications and is used to reduce different variants of the query words to a standard form to avoid the vocabulary mismatch issue in Information Retrieval (IR) systems. Due to specific grammatical rules and complex morphological structures, finding an effective stemming algorithm in Urdu is challenging task. Although, several stemming algorithms have been proposed for the Urdu text stemming; however, none of them extract the stem from multilevel inflected forms. In this context, according to the best of our knowledge, this is a first effort towards the proposition and evaluation of a novel Urdu Text Stemmer (UTS) that can deal with multi-level inflection forms in Urdu text. The experimental evaluation of the proposed scheme has been conducted on the text-based and word-based custom-developed corpus. The proposed stemming technique is rigorously evaluated and compared with state-of-the-art stemming algorithms. Experimental results demonstrate that UTS outperforms existing Urdu stemmers and achieves an accuracy of 94.92% and 91.8% on word corpus and text corpus, respectively. We also evaluated our proposed system in an Information Retrieval application for Urdu, using the Collection for Urdu Retrieval Evaluation (CURE) dataset. Our approach for information retrieval outperformed and improved both recall and precision metrics.

show abstract

Section: Related Workmentioning

confidence: 99%

Building a Multilevel Inflection Handling Stemmer to Improve Search Effectiveness for Urdu Language

Jabbar,

Iqbal,

Alaulamie

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…After reviewing and filtrating the compiled lists, we created a new list of roughly 1,000 domain-independent stop words. Then, we generated their inflected forms following a proposed technique that involves 123 Arabic clitics [18]. As a result, the final list comprises 11,403 stop words.…”

Section: Stop Words Eliminationmentioning

confidence: 99%

The effects of Pre-Processing Techniques on Arabic Text Classification

2021

IJATCSE

View full text Add to dashboard Cite

In the last two decades, the amount of available Arabic text data on the World Wide Web is dramatically growing, making it the fourth most used language on the web. Accordingly, the demand for efficient Arabic text classification is increasing, especially for web page content filtering, information retrieval, and e-mail spam detection. Several Machine Learning algorithms have been implemented to classify Arabic documents. However, the results achieved are not comparable with those obtained in other languages such as English, primarily when using preprocessing techniques that do not take into consideration the Arabic language features. This paper investigates the impact of wisely selected preprocessing techniques on the efficiency of different text classification algorithms. The effects of stop words removal, stemming, lemmatization, and all possible combinations are examined. The reported results (+10.75% to +28.73%) prove the effectiveness of using these techniques either individually or in combination.

show abstract

“…They executed some experiments and evaluated the performance of the developed algorithm based on two styles of Arabic; Classical Arabic and Modern Standard Arabic. The outputs of the stemmer organized in three classes include the stem, a unique root, and a combined class from the root and stem [24].…”

Section: Literature Reviewmentioning

confidence: 99%

“…Heavy/light Stemmer [24] Stemmer algorithm based on Arabic's morphological features. 96.9% Stemming Algorithm [22] Based on a set of rules and a file containing Arabic roots, the authors extract roots from input texts.…”

Section: %mentioning

confidence: 99%

Arabic Text Processing Model: Verbs Roots and Conjugation Automation

2020

View full text Add to dashboard Cite

The Natural Language Processing (NLP) is a process to automate the text or speech of Natural Languages. This automation is mainly conducted for Western languages. The Arabic Language got less focus in this area. This paper presents a Model to recognize an Arabic sentence. A new morphological model based on regular expressions is developed to recognize the Arabic verbs. A hash table containing all Arabic three-letters' root of verbs is implemented. The total number of Arabic verbs that are derived from three-letters' root size is 23090. The number of roots is 6104. A set of rules forming the Arabic grammar is used to derive and analyze the syntax of Arabic sentences. About 87% of the verbs represented in our regular expressions' engine are detected. Moreover, the sentences are also recognized. In several Surat of the Quran, only 9% of the detected verbs are false-positive (a non-verb declared as a verb), and 4% are considered false-negative (a verb is considered as a noun). This rate is mainly because we are not using vowels even that the Quran (our case study) is using them. The reason behind our decision is to be able to handle all Arabic texts, which mostly are not using vowels.

show abstract

Developing and performance evaluation of a new Arabic heavy/light stemmer

Cited by 8 publications

References 24 publications

Building a Multilevel Inflection Handling Stemmer to Improve Search Effectiveness for Urdu Language

Building a Multilevel Inflection Handling Stemmer to Improve Search Effectiveness for Urdu Language

The effects of Pre-Processing Techniques on Arabic Text Classification

Arabic Text Processing Model: Verbs Roots and Conjugation Automation

Contact Info

Product

Resources

About