Cognate Identification to improve Phylogenetic trees for Indian Languages

Kanojia, Diptesh; Kulkarni, Malhar; Bhattacharyya, Pushpak; Haffari, Gholemreza

doi:10.1145/3297001.3297045

Cited by 6 publications

(9 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using false friends as data points with negative labels restricts us to the use of semantic similarity based features, as orthographic or phonetic similarity-based measures would fail to detect sufficient distinction between them. Hence, we use the features proposed by Rama (2016) and Kanojia et al (2019a) as baseline features for a comparative evaluation.…”

Section: Feature Sets For Cognate Detectionmentioning

confidence: 99%

Cognition-aware Cognate Detection

Kanojia

Sharma

Sayali³

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

Self Cite

View full text Add to dashboard Cite

Automatic detection of cognates helps downstream NLP tasks of Machine Translation, Cross-lingual Information Retrieval, Computational Phylogenetics and Cross-lingual Named Entity Recognition. Previous approaches for the task of cognate detection use orthographic, phonetic and semantic similarity based features sets. In this paper, we propose a novel method for enriching the feature sets, with cognitive features extracted from human readers' gaze behaviour. We collect gaze behaviour data for a small sample of cognates and show that extracted cognitive features help the task of cognate detection. However, gaze data collection and annotation is a costly task. We use the collected gaze behaviour data to predict cognitive features for a larger sample and show that predicted cognitive features, also, significantly improve the task performance. We report improvements of 10% with the collected gaze features, and 12% using the predicted gaze features, over the previously proposed approaches. Furthermore, we release the collected gaze behaviour data along with our code and cross-lingual models.

show abstract

Section: Feature Sets For Cognate Detectionmentioning

confidence: 99%

Cognition-aware Cognate Detection

Kanojia

Sharma

Sayali³

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

Self Cite

View full text Add to dashboard Cite

show abstract

“…These cognates can be used to challenge the previously established cognate detection approaches further. Kanojia et al (2019a) perform cognate detection for some Indian languages, but a prominent part of their work includes manual verification and segratation of their output into cognates and noncognates. Identification of cognates for improving IR has already been explored for Indian languages (Makin et al, 2007).…”

Section: Related Workmentioning

confidence: 99%

Challenge Dataset of Cognates and False Friend Pairs from Indian Languages

Kanojia¹,

Bhattacharyya²,

Kulkarni³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Cognates are present in multiple variants of the same text across different languages (e.g., "hund" in German and "hound" in English language mean "dog"). They pose a challenge to various Natural Language Processing (NLP) applications such as Machine Translation, Cross-lingual Sense Disambiguation, Computational Phylogenetics, and Information Retrieval. A possible solution to address this challenge is to identify cognates across language pairs. In this paper, we describe the creation of two cognate datasets for twelve Indian languages, namely Sanskrit, Hindi, Assamese, Oriya, Kannada, Gujarati, Tamil, Telugu, Punjabi, Bengali, Marathi, and Malayalam. We digitize the cognate data from an Indian language cognate dictionary and utilize linked Indian language Wordnets to generate cognate sets. Additionally, we use the Wordnet data to create a False Friends' dataset for eleven language pairs. We also evaluate the efficacy of our dataset using previously available baseline cognate detection approaches. We also perform a manual evaluation with the help of lexicographers and release the curated gold-standard dataset with this paper.

show abstract

“…Ciobanu and Dinu (2014) employ dynamic programming based methods for sequence alignment. Kanojia et al (2019a) perform cognate detection for some Indian languages, but a prominent part of their work includes manual verification and segratation of their output into cognates and non-cognates. Kanojia et al (2019b) utilize recurrent neural networks to harness the character sequence among cognates and non-cognates for Indian languages, but employ monolingual embeddings for the task.…”

Section: Related Workmentioning

confidence: 99%

Harnessing Cross-lingual Features to Improve Cognate Detection for Low-resource Languages

Kanojia

Dabre

Dewangan

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Cognates are variants of the same lexical form across different languages; for example "fonema" in Spanish and "phoneme" in English are cognates, both of which mean "a unit of sound". The task of automatic detection of cognates among any two languages can help downstream NLP tasks such as Cross-lingual Information Retrieval, Computational Phylogenetics, and Machine Translation. In this paper, we demonstrate the use of cross-lingual word embeddings for detecting cognates among fourteen Indian Languages. Our approach introduces the use of context from a knowledge graph to generate improved feature representations for cognate detection. We then evaluate the impact of our cognate detection mechanism on neural machine translation (NMT), as a downstream task. We evaluate our methods to detect cognates on a challenging dataset of twelve Indian languages, namely, Sanskrit, Hindi, Assamese, Oriya, Kannada, Gujarati, Tamil, Telugu, Punjabi, Bengali, Marathi, and Malayalam. Additionally, we create evaluation datasets for two more Indian languages, Konkani and Nepali 1 . We observe an improvement of up to 18% points, in terms of F-score, for cognate detection. Furthermore, we observe that cognates extracted using our method help improve NMT quality by up to 2.76 BLEU. We also release 2 our code, newly constructed datasets and cross-lingual models publicly.

show abstract

Cognate Identification to improve Phylogenetic trees for Indian Languages

Cited by 6 publications

References 10 publications

Cognition-aware Cognate Detection

Cognition-aware Cognate Detection

Challenge Dataset of Cognates and False Friend Pairs from Indian Languages

Harnessing Cross-lingual Features to Improve Cognate Detection for Low-resource Languages

Contact Info

Product

Resources

About