Hierarchical tag-set for rule-based processing of Tamil language

Sarveswaran, Kengatharaiyer; Mahesan, Sinnathamby

doi:10.4038/ijms.v1i2.53

Cited by 5 publications

(3 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In grammar books written by native grammarians (Thesikar 1957;Senavaraiyar 1938) Tamil words have been primarily divided into four types, namely: nouns, verbs-intensifiers/attributives, and particles. However, more recently there have been more granular Part of Speech (POS) analyses proposed by Sarveswaran and Mahesan (2014); Baskaran et al (2008); Lehmann (1993). We follow Lehmann (1993) and Sarveswaran and Mahesan (2014) closely.…”

Section: Part Of Speechmentioning

confidence: 99%

“…However, more recently there have been more granular Part of Speech (POS) analyses proposed by Sarveswaran and Mahesan (2014); Baskaran et al (2008); Lehmann (1993). We follow Lehmann (1993) and Sarveswaran and Mahesan (2014) closely. These are relatively less granular when compared to others, but we have found that these allow for the most accurate analysis in our implementation.…”

Section: Part Of Speechmentioning

confidence: 99%

See 1 more Smart Citation

ThamizhiMorph: A morphological parser for the Tamil language

Sarveswaran

Dias

Butt

2021

Machine Translation

Self Cite

View full text Add to dashboard Cite

This paper presents an open source and extendable Morphological Analyser cum Generator (MAG) for Tamil named ThamizhiMorph. Tamil is a low-resource language in terms of NLP processing tools and applications. In addition, most of the available tools are neither open nor extendable. A morphological analyser is a key resource for the storage and retrieval of morphophonological and morphosyntactic information, especially for morphologically rich languages, and is also useful for developing applications within Machine Translation. This paper describes how ThamizhiMorph is designed using a Finite-State Transducer (FST) and implemented using Foma. We discuss our design decisions based on the peculiarities of Tamil and its nominal and verbal paradigms. We specify a high-level meta-language to efficiently characterise the language’s inflectional morphology. We evaluate ThamizhiMorph using text from a Tamil textbook and the Tamil Universal Dependency treebank version 2.5. The evaluation and error analysis attest a very high performance level, with the identified errors being mostly due to out-of-vocabulary items, which are easily fixable. In order to foster further development, we have made our scripts, the FST models, lexicons, Meta-Morphological rules, lists of generated verbs and nouns, and test data sets freely available for others to use and extend upon.

show abstract

Section: Part Of Speechmentioning

confidence: 99%

Section: Part Of Speechmentioning

confidence: 99%

ThamizhiMorph: A morphological parser for the Tamil language

Sarveswaran

Dias

Butt

2021

Machine Translation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Part of Speech (POS) tagging is an important phase in the parsing process where each word in a sentence is assigned with its POS tag (or lexical category) information. Several attempts have been made to define POS tagsets for Tamil, based on different theories, and level of granularity; (Sarveswaran and Mahesan, 2014) gives an account of different tagsets. Among these, Amrita (Anand Kumar et al, 2010) and BIS 5 are two popular tagsets.…”

Section: Pos Tagging Using Thamizhipostmentioning

confidence: 99%

ThamizhiUDp: A Dependency Parser for Tamil

Sarveswaran¹,

Dias²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

This paper describes how we developed a neural-based dependency parser, namely ThamizhiUDp, which provides a complete pipeline for the dependency parsing of the Tamil language text using Universal Dependency formalism. We have considered the phases of the dependency parsing pipeline and identified tools and resources in each of these phases to improve the accuracy and to tackle data scarcity. ThamizhiUDp uses Stanza for tokenisation and lemmatisation, ThamizhiPOSt and ThamizhiMorph for generating Part of Speech (POS) and Morphological annotations, and uuparser with multilingual training for dependency parsing. ThamizhiPOSt is our POS tagger, which is based on the Stanza, trained with Amrita POS-tagged corpus. It is the current state-of-the-art in Tamil POS tagging with an F1 score of 93.27. Our morphological analyzer, ThamizhiMorph is a rule-based system with a very good coverage of Tamil. Our dependency parser ThamizhiUDp was trained using multilingual data. It shows a Labelled Assigned Score (LAS) of 62.39, 4 points higher than the current best achieved for Tamil dependency parsing. Therefore, we show that breaking up the dependency parsing pipeline to accommodate existing tools and resources is a viable approach for low-resource languages.

show abstract