Implementation of Stemmer and Lemmatizer for a Low-Resource Language—Kannada

Trishala, G.; Mamatha, H. R.

doi:10.1007/978-981-15-8443-5_28

Cited by 3 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ferilli [17] presented a term-document frequency approach that automatically detects stopwords from a small amount of the corpora and stated that it was the most effective approach which outperformed the classic term frequency [18,2,19] and the normalized inverse document frequency of Lo et al [20]. Trishala and Mamatha [21] proposed a rule-based Kannada stemmer relying on an unsupervised approach using k-means algorithm, and Thangarasu and Inbarani [22] presented an analogy removal stemmer that automatically stem Tamil words from the text corpora. The state-of-art of query spelling corrections explained by Chang et al [23] are approaches applicable for low-and high-resource languages.…”

Section: Related Workmentioning

confidence: 99%

Text Information Retrieval in Tetun

Jesús

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Tetun is one of Timor-Leste's official languages alongside Portuguese. It is a low-resource language with over 932,400 speakers that started developing when Timor-Leste restored its independence in 2002. The media mainly uses Tetun, and more than ten national online newspapers actively broadcast news in Tetun every day. However, since information retrieval-based solutions for Tetun do not exist, finding Tetun information on the internet is challenging. This work aims to investigate and develop solutions that can enable the application of information retrieval techniques to develop search solutions for Tetun. We present a preliminary result of an experiment conducted on the task of ad-hoc retrieval in Tetun.

show abstract

Section: Related Workmentioning

confidence: 99%

Text Information Retrieval in Tetun

Jesús

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Document classification is the task of assigning the category 𝑐 𝑖 for the given document 𝑑 𝑗 , where   The figure 1 depicts the generic block diagram of Kannada document classification process. The raw Kannada text documents comprise of some unwanted texts and removal of these is essential for better computation [25]. Stopwords removal, lemmatization, stemming, tokenization, and transliteration are few pre-processing techniques which aide in reducing the dimensionality and complexity of text processing.…”

Section: Generic Architecturementioning

confidence: 99%

“…At the preprocessing stage, lemmatization and stemming are two basic modules used for the normalization of text. In [25] authors presented Unsu-pervised Stemmer and Rule-Based Lemmatizer for Kannada documents. Experimentation is carried out by building a dataset of 17,825 root words with the help of Kannada dictionary.…”

Section: ( ) Log (1)mentioning

confidence: 99%

Semantic Term weighting representation for Kannada Document Classification

Rangan

Harish

2022

Preprint

View full text Add to dashboard Cite

In natural language processing, sequence order of terms plays a vital role. This positional sequence information helps in the semantic analysis of the natural language. The absence of semantic information in term weighting methods motivated us to propose the semantic term weighting representation. On the other hand, to address the demand for Indian regional language resources, especially for the Kannada language we have created an 11,045 Kannada documents dataset. This dataset is multilabel and unbalanced. The proposed semantic term weighting representation methods (Term Frequency-Positional encoding (TF-PE) and Term Frequency-Inverse document frequency-Positional encoding (TF-IDF-PE)) are applied to the proposed dataset. Further, the K-Fold and normal train-test split experimentations are carried out on the proposed dataset. Among the proposed representation methods Unicode encoded Term Frequency-Inverse document frequency-Positional encoding (TF-IDF-PE) representation performed better than Term frequency-Positional encoding (TF-PE) representation. The Unicode encoded TF-IDF-PE representation with the SVM classifier yields better average accuracy of 68.62% in K-10 Fold experimentations.

show abstract