Statistical Part-of-Speech Tagger for Traditional Arabic Texts

Part Of Speech (POS) tagging forms the important preprocessing step in many of the natural language processing applications such as text summarization, question answering and information retrieval system. It is the process of classifying every word in a given context to its appropriate part of speech. Different POS tagging techniques in the literature have been developed and experimented. Currently, it is well known that some POS tagging models are not performing well on the Quranic Arabic due to the complexity of the Quranic Arabic text. This complexity presents several challenges for POS tagging such as high ambiguity, data sparseness and large existence of unknown words. With this in mind, the main problem here is to find out how existing and efficient methods perform in Arabic and how can Quranic corpus be utilized to produce an efficient framework for Arabic POS tagging. We propose a classifiers combination experimental framework for Arabic POS tagger, by selecting two best diverse probabilistic classifiers used in numerous works in non-Arabic language; namely K-Nearest Neighbour (KNN) and Naive Bayes (NB). The Majority voting is used here as the combination strategy to exploit classifiers advantages. In addition, an in-depth study has been conducted on a large list of features for exploiting effective features and investigating their role in enhancing the performance of POS taggers for the Quranic Arabic. Hence, this study aims to efficiently integrate different feature sets and tagging algorithms to synthesize more accurate POS tagging procedure. The data used in this study is the Arabic Quranic Corpus, an annotated linguistic resource consisting of 77,430 words with Arabic grammar, syntax and morphology for each word in the Holy Quran. The highest accuracy in the results achieved is 98.32%, which can be a significant enhancement for the state-of-the-art for Arabic Quranic text. The most effective features that yield this accuracy are a combination of w 0 (the current word), p 0 (POS of the current word), p -3 (POS of three words before), p -2 (POS of two words before) and p -1 (POS of the word before).

show abstract

“…The problems of Arabic studies in POS tagging are as follows (El-Hadj, 2009;Al Gahtani et al, 2009):…”

Section: Pos Tagging Approaches Used For Arabicmentioning

confidence: 99%

Arabic Part of Speech Tagging Using K-Nearest Neighbour and Naive Bayes Classifiers Combination

Mahafdah

Omar

Al-Omari

2014

Journal of Computer Science

View full text Add to dashboard Cite

show abstract

“…Finally, it builds an HMM-based model of Arabic POS tags, which will be trained on the annotated corpus. Alhadj et al [30] propose a new method of part-of-speech tagger that can be used for analyzing and annotating traditional Arabic texts, especially the Holy Quran text.…”

Section: Statistical Approachmentioning

confidence: 99%

Hybrid Part-Of-Speech Tagger for Non-Vocalized Arabic Text

Hadni¹,

Ouatik²,

Lachkar³

et al. 2013

IJNLC

View full text Add to dashboard Cite

Part of speech tagging (POS tagging) has a crucial role in different fields of natural language processing (NLP) including Speech Recognition, Natural Language Parsing, Information Retrieval and Multi Words Term Extraction. This paper proposes an efficient and accurate POS Tagging technique for Arabic language using hybrid approach. Due to the ambiguity issue, Arabic Rule-Based method suffers from misclassified and unanalyzed words. To overcome these two problems, we propose a Hidden Markov Model (HMM) integrated with Arabic Rule-Based method. Our POS tagger generates a set of three POS tags: Noun, Verb, and Particle. The proposed technique uses the different contextual information of the words with a variety of the features which are helpful to predict the various POS classes. To evaluate its accuracy, the proposed method has been trained and tested with two corpora: the Holy Quran Corpus and Kalimat Corpus for undiacritized Classical Arabic language. The experiment results demonstrate the efficiency of our method for Arabic POS Tagging. In fact, the obtained accuracies rates are 97.6%, 96.8% and 94.4% for respectively our Hybrid Tagger, HMM Tagger and for the Rule-Based Tagger with Holy Quran Corpus. And for Kalimat Corpus we obtained 94.60%, 97.40% and 98% for respectively Rule-Based Tagger, HMM Tagger and our Hybrid Tagger.

show abstract

“…Part-Of-Speech (POS) tagging is assigning a specific tag to each word of a sentence to indicate its function in the specific context [1]. POS tagging is considered as one of the basic components necessary for any robust Natural Language Processing (NLP) infrastructure [2], and it is needed in many tasks such as syntax and semantic analysis, text to speech (TTS), natural language parsing, information retrieval (IR), information extraction (IE), and machine translation (MT) [3]. A manually tagged corpus can be used for innumerable studies of word-frequency and POS.…”

Section: Introductionmentioning

confidence: 99%

Lexical and Morphological Statistics of an Arabic POS-Tagged Corpus

Mubarak¹,

Shaban²,

Forat³

2014

The Egyptian Journal of Language Engineering

View full text Add to dashboard Cite

Part-Of-Speech (POS) tagging is a basic component necessary for many Natural Language Processing (NLP) applications. Building a manually tagged corpus helps in studying key statistics of a given language which form the basis for POS tagging systems. In this paper, we present both lexical and morphological statistics for Arabic that are derived from the Sakhr's POS manually tagged corpus. It covers text (7 M words) from a wide range of Arab countries in different domains over the years 2002-2004. The derived statistics are used as heuristics and preferential rules within a statistical Diacritizer which achieves a high accuracy in stem diacritization and POS disambiguation. Statistics includes information related to sentence and word lengths, punctuation marks, distribution of Arabic letters and diacritics, in addition to lexical and morphological information forPOS distribution, stems, prefixes, suffixes, roots, morphological patterns, and morphosyntactic features like gender, number, person, and case ending. Modern Standard Arabic (MSA) is studied by analyzing the coverage of stems, roots, morphological patterns, prefixes, and suffixes. Comparisons with an arbitrary English corpus are shown in applicable cases.

show abstract

Statistical Part-of-Speech Tagger for Traditional Arabic Texts

Cited by 17 publications

References 5 publications

Arabic Part of Speech Tagging Using K-Nearest Neighbour and Naive Bayes Classifiers Combination

Arabic Part of Speech Tagging Using K-Nearest Neighbour and Naive Bayes Classifiers Combination

Hybrid Part-Of-Speech Tagger for Non-Vocalized Arabic Text

Lexical and Morphological Statistics of an Arabic POS-Tagged Corpus

Contact Info

Product

Resources

About