In search of isoglosses: continuous and discrete language embeddings in Slavic historical phonology

Abstract

This paper investigates the ability of neural network architectures to effectively learn diachronic phonological generalizations in a multilingual setting. We employ models using three different types of language embedding (dense, sigmoid, and straight-through). We find that the Straight-Through model outperforms the other two in terms of accuracy, but the Sigmoid model's language embeddings show the strongest agreement with the traditional subgrouping of the Slavic languages. We find that the Straight-Through model has learned coherent, semi-interpretable information about sound change, and outline directions for future research.

Disentangling dialects: a neural approach to Indo-Aryan historical phonology and subgrouping

“…A possible explanation is that by including additional embeddings in our models designed to capture different patterns of sound change in different morphological, semantic and etymological scenarios, we have filtered out critical information relevant to subgrouping, removing valuable genetic signal displayed by morphological traits, which may explain why the model with language embeddings outperforms the other models. A similar negative relationship between model accuracy and genetic signal displayed by embeddings was found by Cathcart and Wandl (2020).…”

“…It is not always straightforward to interpret the sources of differentiation among these embeddings; typically, embeddings based on synchronic patterns of language use in corpora may be due to word order patterns, phonotactic patterns, or a number of other interrelated language-specific distributions. Cathcart and Wandl (2020) investigate the patterns of sound change captured by a neural encoder-decoder architecture trained on Proto-Slavic and contemporary Slavic word forms, and find that embeddings dispay at least partial genetic signal, but also note a negative relationship between overall model accuracy and the degree to which embeddings reflect the communis opinio subgrouping of Slavic.…”

Word prediction in computational historical linguistics

“…In this paper, we investigate the usefulness of word prediction as an intermediate task that may allow us to arrive at computational methods in historical linguistics. The use of word prediction in historical linguistics was first proposed in the first author's master's thesis (Dekker 2018) and independently by Ciobanu and Dinu (2018), followed by recent approaches (List 2019a;Meloni et al 2019;Cathcart and Wandl 2020;Cathcart and Rama 2020;Fourrier and Sagot 2020a). Word prediction is a methodology that enables the use of surface word forms as data (like phenotypic methods), while still capturing the genetic signal through sound correspondences (like genotypic methods), thus allowing for reliable reconstructions of language relationship based on large amounts of data.…”

“…Multiple factors which could lead to an effective use of prediction methods in historical linguistics were evaluated: the choice of machine learning model and encoding of the input data. We evaluated existing models of word prediction (Ciobanu and Dinu 2018;Meloni et al 2019;Cathcart and Wandl 2020;Fourrier [ 321 ] and Sagot 2020a) and came up with our own model, which enables applications on several tasks in historical linguistics. In this paper, we have proposed new approaches for phylogenetic tree reconstruction and cognate detection, based on word prediction error.…”

Rediscovering the Slavic Continuum in Representations Emerging from Neural Models of Spoken Language Identification

“…The most relevant analysis to ours is the recent work by Cathcart and Wandl (2020), in which the authors have trained a neural sequence-to-sequence model on a Slavic etymological dictionary. Their model was trained to consume a reconstructed Proto-Slavic word form and a language embedding, then emit a word form in the modern language specified by the language embedding.…”