N-gram and Local Context Analysis for Persian text retrieval

AleAhmad, Abolfazl; Hakimian, Pantea; Mahdikhani, F.; Oroumchian, Farhad

doi:10.1109/isspa.2007.4555345

Cited by 22 publications

(12 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TNT POS tagger was trained on Bijankhan POS collection with 40 tags. Subsequently the Hamshahri corpus [19] and its CLEF queries were tagged using this tagger (see ). After experimenting with different tagging schemas, the corpus and the queries were stemmed in order to evaluate the effect of stemming and its interaction with POS tagging in retrieval context.…”

Section: Methodology and Implementationmentioning

confidence: 99%

Improving Persian Information Retrieval Systems Using Stemming and Part of Speech Tagging

Karimpour

Ghorbani

Pishdad

et al. 2009

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Section: Methodology and Implementationmentioning

confidence: 99%

Improving Persian Information Retrieval Systems Using Stemming and Part of Speech Tagging

Karimpour

Ghorbani

Pishdad

et al. 2009

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

“…The N-gram is a language-independent approach in which each word is broken down into substrings of length N. This approach has been applied to information retrieval in many languages such as English [9], Turkish [8], Malay [16] and Farsi [1] with varying degrees of success. In [13], Larkey et al have used the bigram and trigram string similarity approach for Arabic text retrieval.…”

Section: Related Researchmentioning

confidence: 99%

“…The N-gram approach has mixed performances in information retrieval. In some languages like English, it results in a poor performance however in languages like Farsi, it has an acceptable performance [1]. As mentioned earlier, most Arabic words are made up of roots with three letters which led us to use trigrams for word segmentation.…”

Section: N-gram Conflation and Co-occurrence Analysis Formentioning

confidence: 99%

Corpus-Based Arabic Stemming Using N-Grams

Zitouni

Damankesh

Barakati

et al. 2010

Information Retrieval Technology

Self Cite

View full text Add to dashboard Cite

Abstract. In languages with high word inflation such as Arabic, stemming improves text retrieval performance by reducing words variants. We propose a change in the corpus-based stemming approach proposed by Xu and Croft for English and Spanish languages in order to stem Arabic words. We generate the conflation classes by clustering 3-gram representations of the words found in only 10% of the data in the first stage. In the second stage, these clusters are refined using different similarity measures and thresholds. We conducted retrieval experiments using row data, Light-10 stemmer and 8 different variations of the similarity measures and thresholds and compared the results. The experiments show that 3-gram stemming using the dice distance for clustering and the EM similarity measure for refinement performs better than using no stemming; but slightly worse than Light-10 stemmer. Our method potentially could outperform Light-10 stemmer if more text is sampled in the first stage.

show abstract

“…For example if we consider an English query that has three terms then the most probable Persian translation of the query terms would be E[1,TDimes [1,1]], E[2,TDimes [1,2]] and E[3,TDimes [1,3]] respectively and the translated query's weight would be CTP[TopColumns [1],TopRows [1]]. …”

Section: That Correspond To Top N Translations Of the Query Q = {Q I mentioning

confidence: 99%

Experiments with English-Persian text retrieval

AleAhmad

Amiri

Rahgozar

et al. 2008

Proceedings of the 2nd ACM Workshop on Improving Non English Web Searching

Self Cite

View full text Add to dashboard Cite

As the number of non-English documents is increasing dramatically on the web nowadays, the study and design of information retrieval systems for these languages is very important. The Persian language is the official language of Iran, Afghanistan and Tajikistan and is also spoken in some other countries in the Middle East, so there are significant amount of Persian documents available on the web. In this study, we will present and compare our English-Persian cross language text retrieval experiments on Hamshahri text collection. Also, we will present Combinatorial Translation Probability (CTP) calculation method for query translation that estimates translation probabilities based on the collection itself.

show abstract

N-gram and Local Context Analysis for Persian text retrieval

Abstract: The Persian language is one of the languages in MiddleEast, so

Cited by 22 publications

References 9 publications

Improving Persian Information Retrieval Systems Using Stemming and Part of Speech Tagging

Improving Persian Information Retrieval Systems Using Stemming and Part of Speech Tagging

Corpus-Based Arabic Stemming Using N-Grams

Experiments with English-Persian text retrieval

Contact Info

Product

Resources

About