Instant annotations in ELAN corpora of spoken and written Komi, an endangered language of the Barents Sea region

Abstract

The paper describes work-in-progress by the Izhva Komi language documentation project, which records new spoken language data, digitizes available recordings and annotate these multimedia data in order to provide a comprehensive language corpus as a databases for future research on and for this endangered -and under-described -Uralic speech community. While working with a spoken variety and in the framework of documentary linguistics, we apply language technology methods and tools, which have been applied so far only to normalized written languages. Specifically, we describe a script providing interactivity between ELAN, a Graphical User Interface tool for annotating and presenting multimodal corpora, and different morphosyntactic analysis modules implemented as Finite State Transducers and Constraint Grammar for rule-based morphosyntactic tagging and disambiguation. Our aim is to challenge current manual approaches in the annotation of language documentation corpora.

The First Komi-Zyrian Universal Dependencies Treebanks

“…The Northern Saami and Erzya corpora, for example, seem to have been created using a similar approach. Some work has been conducted with integrating these NLP tools into workflows commonly used in language documentation (Gerstenberger et al, 2017a(Gerstenberger et al, ,b, 2016. Since these languages often lack larger annotated resources, the use of infrastructures other than rule-based ones has not been common or possible, but these workflows have been implemented in a modular fashion that would make enable the integration of other tools when they become available or reach needed accuracy.…”

Building a Morphological Analyser for Laz

“…By the end of this century, many will not survive with the decreasing number of the native speakers of such languages (Riza, 2008). This has alarmed not only native speakers of these languages but also research community to direct their attention for language documentation as well as preservation and revitalization studies for these languages (Ćavar et al, 2016;Gerstenberger et al, 2017). Bird (2009) calls out for a 'new kind of computational linguistics' in his paper that would protect this endangered invaluable cultural heritage by helping to accelerate these studies, and he ends his paper with these words 'Who knows, we may even postpone the day when these languages utter their last words.'…”

“…However, with little data at hand these methods may not present a good solution. Therefore, Gerstenberger et al (2017) suggests a rulebased morpho-syntactic modelling for annotating small language data. On their study of Komi language, his results show by-far significant advantages of rule-based approaches for endangered languages by providing much more precise results in tagging as well as 'full-fledged grammatical description based on broad empirical evidence' and a future development for computer-assisted language learning systems.…”

“…In this study, the aim is to create a morphological analyzer using the Helsinki Finite State Toolkit (HFST) that will help to overcome manual annotation of a potential Laz corpus. Additionally, as Gerstenberger et al (2017) suggest, these may later help developing programs to be able to facilitate learning of Laz, considering the increase of interest in Laz courses not only in secondary schools but also in universities such as Boğaziçi University and İstanbul Bilgi University (Haznedar, 2018). From spelling and grammar-checkers to machine translation systems and language learning materials, this small study will hopefully lead to further developments on Laz language in the field of Computational Linguistics.…”

Dependency Parsing of Code-Switching Data with Cross-Lingual Feature Representations

“…Computational linguistic research on Komi is so far only in a development stage. However, an FST morphological analyzer and a (rudimentary) syntactic parser based on Constraint Grammar are available at Giellatekno/Divvun -Saami Language Technology at UiT The Arctic University of Norway¹ and work on a complete Constraint Grammar description to be implemented into a rule-based syntactic parser is currently carried out in collaboration by Giellatekno, the Izhva Komi Documentation Project Gerstenberger et al (2016Gerstenberger et al ( , 2017 and FU-Lab², which has also created a written Komi National corpus (with over 30M words), free electronic dictionaries and a Hunspell checker (including morpheme lists).…”