Stemming Versus Light Stemming as Feature Selection Techniques for Arabic Text Categorization

Duwairi, Rehab; Alrefai, Mu'ath; Khasawneh, Natheer

doi:10.1109/iit.2007.4430403

Cited by 40 publications

(23 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Arabs also use dialect instead of MSA in important fields 22 such as online communication (chat rooms,SMS,Facebook,23 Twitters and others). Most of the research on Arabic is focused 24 on MSA (Duwairi et al, 2007;Harrag et al, 2011;AI-Shalabi 25 et al, 2003;Goweder et al, 2008). Currently, there are 12 26 different Arabic dialects spoken in 28 countries around the 27 world.…”

mentioning

confidence: 98%

A rule-based stemmer for Arabic Gulf dialect

Abuata

Al-Omari

2015

Journal of King Saud University - Computer and Information Scie

View full text Add to dashboard Cite

Arabic dialects arewidely used from many years ago instead of Modern Standard Arabic language in many fields. The presence of dialects in any language is a big challenge. Dialects add a new set of variational dimensions in some fields like natural language processing, information retrieval and even in Arabic chatting between different Arab nationals. Spoken dialects have no standard morphological, phonological and lexical like Modern Standard Arabic. Hence, the objective of this paper is to describe a procedure or algorithm by which a stem for the Arabian Gulf dialect can be defined. The algorithm is rule based. Special rules are created to remove the suffixes and prefixes of the dialect words. Also, the algorithm applies rules related to the word size and the relation between adjacent letters. The algorithm was tested for a number of words and given a good correct stem ratio. The algorithm is also compared with two Modern Standard Arabic algorithms. The results showed that Modern Standard Arabic stemmers performed poorly with Arabic Gulf dialect and our algorithm performed poorly when applied for Modern Standard Arabic words. Ó 2015 Production and hosting by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

show abstract

mentioning

confidence: 98%

A rule-based stemmer for Arabic Gulf dialect

Abuata

Al-Omari

2015

Journal of King Saud University - Computer and Information Scie

View full text Add to dashboard Cite

show abstract

“…Here we note that light stemming maintains the difference between ( ‫الكاتثون‬ ‫الكتاب‬ ) which means "the book" and "the writers" respectively; their light stems are ( ‫كتاتي‬ ‫)كاتة‬ which means book and writer. [12] …”

Section: B Light Stemmingmentioning

confidence: 99%

“…Applying stemming algorithms as a feature selection method reduces the number of features since lexical forms (of words) are derived from basic building blocks; and hence, many features that are generated from the same stem are represented as one feature (their stem). [12] Two types of stemming will be applied to Arabic documents in addition towithout stemming type: a. Root-based Stemming Stemming using root extractor which uses morphological analysis for Arabic words, Figure 2 depicts an example of using stemming for feature selection. Note that several words such as ( ‫المكتثة‬ ‫الكاتة‬ ‫)الكتاب‬ which mean "the library", "the writer" and "the book" respectively are reduced to one stem ‫)كتة(‬ which means write [ 13] as shown in figure 3 [9] which describes preprocessing steps in root based stemming.…”

Section: Arabic Text Pre-processingmentioning

confidence: 99%

Stemming Effectiveness in Clustering of Arabic Documents

Ghanem¹,

Ashour²

2012

IJCA

View full text Add to dashboard Cite

Clustering is an important task gives good results with information retrieval (IR), it aims to automatically put similar documents in one cluster. Stemming is an important technique, used as feature selection to reduce many redundant features have the same root in root-based stemming and have the same syntacticalform in light stemming. Stemming has many advantages it reducesthe size of document and increases processing speed and used in many applications as information retrieval (IR). In this paper, we have evaluatedstemmingtechniques in clustering of Arabic language documents and determined the most efficient in preprocessing of Arabic language,whichis more complex than most other languages. Evaluation used three stemming techniques: root-based Stemming, light Stemming and without stemming. K-means, one of famous and widely clustering algorithm, is applied for clustering.Evaluation depends on recall, precision andF-measure methods. From experiments, results show that light stemming achieved best results in terms of recall, precision and F-measure when compared with others stemming.

show abstract

“…Removing stop words as mentioned before is one way to eliminate unimportant features [1]. Computing term-goodness based on the statistical characteristics of the dataset such as document frequency, information gain, and mutual information is another way [10]. A threshold method, as a method of feature selection is based on removing some features, the removal will be based on the frequencies of those features by setting that frequencies be greater than or less than a defined threshold value.…”

Section: Feature Selection and Reductionmentioning

confidence: 99%

“…Their results show that Rocchio classifier performs better than k-nearest neighbor classifier. Another study conducted in [10] used stemming and light stemming techniques as feature selection techniques, K-nearest neighbors (KNN) as a classifier. Results reported indicated that light stem was superior over stemming in terms of classifier accuracy.…”

Section: Introductionmentioning

confidence: 99%

Classifying Arabic Text Using KNN Classifier

Al-Badarenah¹,

Al-Shawakfa²,

Al-Rababah³

et al. 2016

ijacsa

View full text Add to dashboard Cite

Abstract-With the tremendous amount of electronic documents available, there is a great need to classify documents automatically. Classification is the task of assigning objects (images, text documents, etc.) to one of several predefined categories. The selection of important terms is vital to classifier performance, feature set reduction techniques such as stop word removal, stemming and term threshold were used in this paper. Three term-selection techniques are used on a corpus of 1000 documents that fall in five categories. A comparison study is performed to find the effect of using full-word, stem, and the root term indexing methods. K-nearest -neighbors classifiers used in this study. The averages of all folds for Recall, Precision, Fallout, and Error-Rate were calculated. The results of the experiments carried out on the dataset show the importance of using k-fold testing since it presents the variations of averages of recall, precision, fallout, and error rate for each category over the 10-fold.

show abstract

Stemming Versus Light Stemming as Feature Selection Techniques for Arabic Text Categorization

Cited by 40 publications

References 6 publications

A rule-based stemmer for Arabic Gulf dialect

A rule-based stemmer for Arabic Gulf dialect

Stemming Effectiveness in Clustering of Arabic Documents

Classifying Arabic Text Using KNN Classifier

Contact Info

Product

Resources

About