Word prediction in computational historical linguistics

Abstract

In this paper, we investigate how the prediction paradigm from machine learning and Natural Language Processing (NLP) can be put to use in computational historical linguistics. We propose word prediction as an intermediate task, where the forms of unseen words in some target language are predicted from the forms of the corresponding words in a source language. Word prediction allows us to develop algorithms for phylogenetic tree reconstruction, sound correspondence identification and cognate detection, in ways close to attested methods for linguistic reconstruction. We will discuss different factors, such as data representation and the choice of machine learning model, that have to be taken into account when applying prediction methods in historical linguistics. We present our own implementations and evaluate them on different tasks in historical linguistics.

A New Framework for Fast Automated Phonological Reconstruction Using Trimmed Alignments and Sound Correspondence Patterns

“…The new framework has the advantage of being easy to use, easy to extend, and fast to apply, while at the same time yielding promising results on a newly compiled collection of datasets from three different languages families. Given that our framework can be easily extended, by varying the individual components of the worfklow, we hope that it will provide a solid basis for future work on phonological reconstruction, as well as the prediction of words from cognate reflexes (Bodt and List, 2022;Dekker and Zuidema, 2021;Beinborn et al, 2013;Fourrier et al, 2021) in computational historical linguistics.…”

Probing Multilingual Cognate Prediction Models

“…Automatic cognate prediction has been studied using character-level machine translation techniques (Beinborn et al, 2013;Wu and Yarowsky, 2018;Dekker, 2018;Hämäläinen and Rueter, 2019;Four- rier and Sagot, 2020a). Dekker and Zuidema (2021) provide an overview of the different neural approaches used to solve this task (including their own), as well as its applications to other historical linguistic tasks (such as phylogeny reconstruction). However, the current paper follows specifically the tracks of two previous works studying encoderdecoder models for Romance cognate prediction.…”

“…The cognate prediction task aims at predicting, from a phonetised word, the plausible phonetic form of its cognate in a related language, according to known sound correspondence patterns; this has many applications, from identifying new words with field linguists (Bodt et al, 2018;Bodt and List, 2019) to inducing translation lexicons for lowresourced languages (Mann and Yarowsky, 2001). 3 This task has been modelled as a sequence to sequence character level machine translation task in the most recent papers studying it (see the survey on cognate prediction in Dekker and Zuidema (2021)), which drew linguistic conclusions on the latent information learnt by such models by studying their outputs in a 'black-box' fashion. However, no paper that we know of tried to confirm or inform these conclusions by using modern interpretability tools, such as probing tasks, hidden representation analysis, or inner components analysis.…”

Approaching Reflex Predictions as a Classification Problem Using Extended Phonological Alignments

“…The task can be seen as a form of zero-shot learning (Xian et al, 2018), where a model must learn to predict the "reflexes" of a potentially unknown ancestral word form, with no examples of the relevant cognate set provided during the training phase. When considering the landscape of machine learning methods available and the approaches so far proposed (Dinu and Ciobanu, 2014;Bodt and List, 2022;Meloni et al, 2021;Beinborn et al, 2013;Dekker and Zuidema, 2021;Fourrier et al, 2021;List et al, forthcoming(b)), including other submissions to this challenge (Jäger, 2022;Celano, 2022;Kirov et al, 2022), it is possible to identify two main strategies for the task. The first one treats the problem as one of classification, potentially refining sequence results with probabilities from a character model, while the second employs sequence transformation methods, especially those akin to seq2seq approaches (Sutskever et al, 2014), making the task one analogous to that of "translation".…”