Design and Development of a Stemmer for Punjabi

Abstract

Stemming is the process of removing the affixes from inflected words, without doing complete morphological analysis. A stemming Algorithm is a procedure to reduce all words with the same stem to a common form [20]. It is useful in many areas of computational linguistics and information-retrieval work. This technique is used by the various search engines to find the best solution for a problem. The algorithm is a basic building block for the stemmer. Stemmer is basically used in information retrieval system to improve the performance .The paper present a stemmer for Punjabi, which uses a brute force algorithm. We also use a suffix stripping technique in our paper. Similar techniques can be used to make stemmer for other languages such as Hindi, Bengali and Marathi. The result of stemmer is good and it can be effective in information retrieval system. This stemmer also reduces the problem of over-stemming and under-stemming.

Review on Stemming Techniques

“…Stemming can be used for indexing and search system. In order to develop any application in NLP like text extraction, machine interpretation, document arrangement, topic tracking, text outline, etc., stemmer is required as a basic linguistic resource for any language in the world to attain high accuracy [5].…”

“…The authors reported the accuracy of stemmer is 87.37%. Kumar and Rana (2011) [5] presented a design and development stemmer for Punjabi language, which used a brute force and suffix stripping technique. Brute force does not require text preprocessing.…”

Development of Morphological Stemmer, Analyzer and Generator for Kannada Nouns

“…Most of them are suitable for information retrieval system [4]. Some of the Indian stemmers designed for information retrieval system are as follows: A lightweight Stemmer for Hindi [5], Assas-Band, an Affix-ExceptionList Based Urdu Stemmer [6], Design and Development of a Stemmer for Punjabi [7], and Hybrid Stemmer for Gujarati [8]. There are few stemmers for MTSs [9], they split the given word into stem and suffixes, but they do not provide morphological information about the given word.…”