A Rule-Based Approach to Identify Stop Words for Gujarati Language

Rakholia, Rajnish M.; Saini, Jatinderkumar R.

doi:10.1007/978-981-10-3153-3_79

Cited by 25 publications

(13 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generation of stop words in different Indian languages is also an evolving area [19] [22]. Different Indian languages [20] such as Hindi [27][28] [29] are explored by different researchers and NLP elements explored for each language are stated.…”

Section: Different Types Of Stemming Techniques For Indian Andmentioning

confidence: 99%

Marathi Document: Similarity Measurement using Semantics-based Dimension Reduction Technique

Bafna¹,

Jatinderkumar²

2020

IJACSA

View full text Add to dashboard Cite

Textual data is increasing exponentially and to extract the required information from the text, different techniques are being researched. Some of these techniques require the data to be presented in the tabular or matrix format. The proposed approach designs the Document Term Matrix for Marathi (DTMM) corpus and converts unstructured data into a tabular format. This approach has been called DTMM in this paper and it fails to consider the semantics of the terms. We propose another approach that forms synsets and in turn reduces dimensions to formulate a Document Synset Matrix for Marathi (DSMM) corpus. This also helps in better capturing the semantics and hence is context-based. We abbreviate and call this approach as DSMM and carry out experiments for documentsimilarity measurement on a corpus consisting of more than 1200 documents, consisting of both verses as well as proses, of Marathi language of India. Marathi text processing has been largely an untouched area. The precision, recall, accuracy, F1-score and error rate are used to prove the betterment of the proposed technique.

show abstract

Section: Different Types Of Stemming Techniques For Indian Andmentioning

confidence: 99%

Marathi Document: Similarity Measurement using Semantics-based Dimension Reduction Technique

Bafna¹,

Jatinderkumar²

2020

IJACSA

View full text Add to dashboard Cite

show abstract

“…For text analysis of Indian languages, Punjabi has been explored through stop words identification [22] and categorization [23] as well as poetry corpus creation [24] and classification [25][26]. Gujarati has been, similarly explored through diacritic extraction technique [27], information retrieval [28], stop words identification [29] and categorization [30], Machine Translation System (MTS) [31][32] and classification [33]. Sanskrit has been explored through stop word generation [34] and analysis [35], bilingual dictionary [36], constituency mapper [37], lemmatizer development [38] and comparison of its morphological analyzers [39].…”

Section: Literature Reviewmentioning

confidence: 99%

Marathi Text Analysis using Unsupervised Learning and Word Cloud

Bafna¹,

Saini²

2020

IJEAT

Self Cite

View full text Add to dashboard Cite

Managing a large number of textual documents is a critical and significant task and supports many applications ranging from information retrieval to clustering search engine results. Marathi is one of the oldest of the regional languages in the Indo-Aryan language family, dating from about AD 1000. Abundance of Marathi literature has generated a big corpus and need of summarization of information. The objective of this study is to overcome the scalability problem while managing the documents and summarize the Marathi corpus by extracting tokens. The work is better in terms of scalability and supports the consistent quality of cluster for incremental data set. Most of the past and contemporary research works have targeted English corpus document management. Marathi corpus has been mostly exploited by the researchers for exploring stemming, single-document summarization and classifier design on Marathi corpus. Implementing unsupervised learning on the Marathi corpus for summarization of multiple documents through Word Cloud is still an untouched area. Technically speaking, the current work is an application of TF-IDF, cosine-based document similarity measures and cluster dendrograms, in addition to various other Natural Language Processing (NLP) activities. Entropy and precision are used to evaluate the experiments carried on different datasets and results prove the robustness of the proposed approach for Marathi Corpus.

show abstract

“…Vijayarani S, et al [7] used Zipf's Law (Z method) for creation of stop-words. Rakholia and Saini [8] have presented a rule based approach to dynamically identify stop words for Gujarati language. Vandana Jha et al [9] developed an algorithm to remove stopwords from the Hindi text based on Deterministic finite automata.…”

Section: Related Workmentioning

confidence: 99%

Automatic Generation of Stopwords in the Amharic Text

Miretie¹,

Khedkar²

2018

IJCA

View full text Add to dashboard Cite

For the retrieval of information from documents of different natural languages, pre-processing of the document is the main task. During pre-processing, words which occur too frequently and have little semantic in the document should be identified. Such words are called Stopwords. Stopwords list for different world languages like English, Chinese, Hindi, Arabic Sanskrit etc. are identified. But as I long as I know there is no standard method to identify these words for the Amharic language. In this paper, we proposed the automatic identification of Stopwords for the Amharic text by an aggregate based methodology of words frequency, inverse document frequency, and entropy value measure. Available works on Stopwords identification techniques are based on static or dictionary based Stopwords lists. This method inefficient and very expensive and it is a time-consuming task as the searching process takes a long time. The proposed work will overcome these problems using aggregated methods of both frequency measures and entropy measures of words in the Amharic text for the automatic Stopwords identification.

show abstract

A Rule-Based Approach to Identify Stop Words for Gujarati Language

Cited by 25 publications

References 6 publications

Marathi Document: Similarity Measurement using Semantics-based Dimension Reduction Technique

Marathi Document: Similarity Measurement using Semantics-based Dimension Reduction Technique

Marathi Text Analysis using Unsupervised Learning and Word Cloud

Automatic Generation of Stopwords in the Amharic Text

Contact Info

Product

Resources

About