Ve’rdd. Narrowing the Gap between Paper Dictionaries, Low-Resource NLP and Community Involvement

Alnajjar, Khalid; Hämäläinen, Mika; Rueter, Jack; Partanen, Niko

doi:10.18653/v1/2020.coling-demos.1

Cited by 7 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…At the current, stage our dictionary editing system, Ve rdd [4,3], contains words for multiple endangered languages and their translations in a graph structure. This data could be extended by predicting new relations into the graph with semantic models such as word embeddings.…”

Section: Discussionmentioning

confidence: 99%

“…In the UD we are dealing with, translation sentences might appear in the comments. The UD of the endangered languages can be obtained directly from Universal Dependencies' website 4 . At the time of writing, 1,690, 167, 104 and 435 sen-tences were in Erzya's [44], Moksha's [42], Skolt Sami's [30] and Komi-Zyrian's UDs 5 [32], respectively.…”

Section: Universal Dependenciesmentioning

confidence: 99%

See 1 more Smart Citation

When Word Embeddings Become Endangered

Alnajjar

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Big languages such as English and Finnish have many natural language processing (NLP) resources and models, but this is not the case for low-resourced and endangered languages as such resources are so scarce despite the great advantages they would provide for the language communities. The most common types of resources available for low-resourced and endangered languages are translation dictionaries and universal dependencies. In this paper, we present a method for constructing word embeddings for endangered languages using existing word embeddings of different resource-rich languages and the translation dictionaries of resource-poor languages. Thereafter, the embeddings are fine-tuned using the sentences in the universal dependencies and aligned to match the semantic spaces of the big languages; resulting in crosslingual embeddings. The endangered languages we work with here are Erzya, Moksha, Komi-Zyrian and Skolt Sami. Furthermore, we build a universal sentiment analysis model for all the languages that are part of this study, whether endangered or not, by utilizing cross-lingual word embeddings. The evaluation conducted shows that our word embeddings for endangered languages are well-aligned with the resource-rich languages, and they are suitable for training task-specific models as demonstrated by our sentiment analysis model which achieved a high accuracy. All our cross-lingual word embeddings and the sentiment analysis model have been released openly via an easy-to-use Python library.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Universal Dependenciesmentioning

confidence: 99%

When Word Embeddings Become Endangered

Alnajjar

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our system Ve rdd [1] was the reason I got an opportunity to visit the Sami Culture Center Sajos 5 in Inari, Finland to collaborate with two Skolt Sami dictionary editors. Skolt Sami (sms) is a severely endangered language with only 300 native speakers according to UNESCO.…”

Section: Endangered But How Endangered?mentioning

confidence: 99%

Endangered Languages are not Low-Resourced!

Hämäläinen

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The term low-resourced has been tossed around in the field of natural language processing to a degree that almost any language that is not English can be called "low-resourced"; sometimes even just for the sake of making a mundane or mediocre paper appear more interesting and insightful. In a field where English is a synonym for language and low-resourced is a synonym for anything not English, calling endangered languages low-resourced is a bit of an overstatement. In this paper, I inspect the relation of the endangered with the low-resourced from my own experiences.

show abstract

“…It is noteworthy, that mostly, in lexicography and grammar, tests, are nothing more than example word forms and their preferred annotations, be it grammatical analysis or perhaps spelling corrections, this, naturally, is in and of itself interesting for a linguist even without the testing feature. As a recent feature for the documentation comments, the output format has been updated to GitHub's github-pages format and a prototypical examples can be found at the time of writing in the GiellaLT github space 18 .…”

Section: Documentationmentioning

confidence: 99%