Implementation and Analysis of Different POS Tagger in Khasi Language

Abstract

Part-of-speech tagging is a process of assigning each word of a sentence to a part of speech based on its context and definition. POS tagging is a prerequisite tool for many NLP tasks like word sense disambiguity, name entity information extraction, etc. Unfortunately, very little work has been done so far in this line for Khasi Language. The main difficulty lies with the unavailability of an annotated corpus. Hence, a small corpus is created which consists of 778 sentences with 34,873 words, out of which 3,942 are distinct words and a tagset of 52 tags. In this chapter, three methods for POS tagging, namely Brill's tagger, hidden Markov model (HMM)-based tagger, and bidirectional long short-term memory recurrent neural network (Bi-LSTM), have been implemented. Then a comparative analysis is performed, and it is observed that Bi-LSTM performs better in terms of accuracy.

“Money Doesn’t Like Noise”

“…To aid the identification of attitude markers, the corpus was grammatically annotated using part of speech tagging (POS-tag). The process of assigning tags to words in a sentence, such as noun, verb, adjective, and adverb, is known as part-of-speech (POS) tagging (Hynniewta, Maji, & Warjri, 2022).…”