Development of Marathi part of speech tagger using statistical approach

Abstract

Part-of-speech (POS) tagging is a process of assigning the words in a text corresponding to a particular part of speech. A fundamental version of POS tagging is the identification of words as nouns, verbs, adjectives etc. For processing natural languages, Part of Speech tagging is a prominent tool. It is one of the simplest as well as most constant and statistical model for many NLP applications. POS Tagging is an initial stage of linguistics, text analysis like information retrieval, machine translator, text to speech synthesis, information extraction etc. In POS Tagging we assign a Part of Speech tag to each word in a sentence and literature. Various approaches have been proposed to implement POS taggers. In this paper we present a Marathi part of speech tagger. It is morphologically rich language. Marathi is spoken by the native people of Maharashtra. The general approach used for development of tagger is statistical using Unigram, Bigram, Trigram and HMM Methods. It presents a clear idea about all the algorithms with suitable examples. It also introduces a tag set for Marathi which can be used for tagging Marathi text. In this paper we have shown the development of the tagger as well as compared to check the accuracy of taggers output. The three Marathi POS taggers viz. Unigram, Bigram, Trigram and HMM gives the accuracy of 77.38%, 90.30%, 91.46% and 93.82% respectively.

POS Tagging Approaches: A Comparison

“…The approaches of POS tagging can be divided into three categories; rule based tagging, statistical tagging and hybrid tagging [1]. A set of hand written rules are applied along with it the contextual information is used to assign POS tags to words in the rule based POS.…”

“…The best tag can be determined by it for a word by finding out the probability that it occurs with the n previous tags, where the value of n is set to 1, 2 or 3 for practical purposes. These models are termed as Unigram, Bigram and Trigram [1]. Viterbi algorithm, which is a search algorithm that avoids the polynomial expansion of a breadth first search by trimming the search tree at each level using the best m Maximum Likelihood Estimates (MLE).…”

“…POS tagging makes up a primal task for processing a natural language. It is built up using linguistic theory rule, random pattern and sometimes a combining both [1]. My work shows the evolution of an easy and effective automatic tagger in support of inflectional and derivational morphologically rich language Hindi.…”

“…Part of speech tagging is very significant pre-processing task for Natural language processing activities [1]. A Part of speech (POS) tagger has been developed in order to check off the words and punctuation in a textual matter having suitable POS labels of Hindi text.…”

A Survey on Parts of Speech Tagging for Indian Languages

“…Supervised POS taggers are based on pre-tagged corpora [6], which are used for training to learn information about the word-tag frequencies, rule and tag set, sets etc. The performance of the models generally increases with the increase in size of these corpora.…”