Normalized dataset for Sanskrit word segmentation and morphological parsing

Krishnan, Sriram; Kulkarni, Amba; Huet, Gérard

doi:10.1007/s10579-024-09724-0

Lang Resources & Evaluation

2024

DOI: 10.1007/s10579-024-09724-0

|View full text |Cite

Normalized dataset for Sanskrit word segmentation and morphological parsing

Sriram Krishnan,

Amba Kulkarni,

Gérard Huet

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Preprint1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Optimizing Urdu Text Tokenization: Morphological Rules for Compound Word Identification

Khushhal,

Majid,

Babar

et al. 2024

Preprint

View full text Add to dashboard Cite

Tokenization in a text document is regarded as a primary natural language processing task for feature generation, and it plays a vital role in sentiment analysis, information retrieval, part of speech tagging, and named entity recognition. Urdu is spoken by around 170.2 million people worldwide as their first or second language. It is a morphologically and orthographically rich language. Word tokenization in Urdu text documents is very challenging because word boundaries are not specified by only space, as in other languages. A compound, a multi-word expression, is a more complex word consisting of multiple strings or independent base words. Tokens are the minimal unit of any language with a suitable semantic structure. Traditionally, bigram or trigram approaches represent compound words in the tokenization process. This research proposes a morphological rules-based approach to identify compound words in Urdu text for tokenization. A thorough evaluation is performed on a dataset of reasonable size to compare the performance of the proposed technique with traditional approaches. Results show that the proposed method can accurately identify the compound words for the tokenization of Urdu text documents. Notably, using morphological rule-based techniques for compound words reduces the number of extracted features.

show abstract

Optimizing Urdu Text Tokenization: Morphological Rules for Compound Word Identification

Khushhal,

Majid,

Babar

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Normalized dataset for Sanskrit word segmentation and morphological parsing

Cited by 1 publication

References 25 publications

Optimizing Urdu Text Tokenization: Morphological Rules for Compound Word Identification

Optimizing Urdu Text Tokenization: Morphological Rules for Compound Word Identification

Contact Info

Product

Resources

About