Variability-based neighbor clustering: A bottom-up approach to periodization in historical linguistics

Abstract

One of the linguistic sub-disciplines that has benefited most from corpora is historical linguistics, and many new diachronic resources have become available especially during the last ten to fifteen years. However, the longitudinal nature and the more constrained sampling of diachronic corpora present a number of challenges for historical linguists. One central problem that has received little attention is the periodization of a linguistic phenomenon P; that is, how the development of P over time can be divided into periods or stages. Because neither the year-by-year development of P nor the pre-defined historical periods distinguished by corpus compilers may be particularly meaningful or revealing for P, a bottom-up approach might be more appropriate for determining the stages of P’s development. However, this is difficult to accomplish using existing methods. This article describes a new clustering method, “Variability-based Neighbor Clustering” (VNC), to identify periods in the historical development of P that accounts for the temporal ordering of the data.

From movement into action to manner of causation: changes in argument mapping in the into-causative

“…The variable PERIOD represents larger temporal bins, which were determined by Variability-based Neighbor Clustering (VNC; Gries and Hilpert 2012). VNC clusters adjacent decades in a bottom-up fashion, based on their similarity in relative frequency, in order to avoid arbitrary periodization.…”

Comparative sociolinguistic perspectives on the rate of linguistic change

“…3Descriptive empirical research shows the extent to which the trajectories of individual changes can alternate between phases of rapid, abrupt change and more or less stable frequency plateaus. This variability may correspond to data granularity, on the one hand, reflecting the structure and coverage of the database, and the quantitative methods and statistical models adopted (Gries and Hilpert 2012;Nevalainen 2015b). Importantly, however, the observed variation can be a reflection of various language-external influences, which impact on the pace of linguistic change either in the short term or over a longer period of time.…”

Studying the history of the Arabic language: language technology and a large-scale historical corpus

“…BoilerP lates ← Conflate(P hrases) 10 return Boilerplates 11 Function FindFrequentPhrases(T exts) 12 Initialize dictionary P hrases(phrase, counter) 13 for i ← 1 to T do BoilerP lates ← FindBoilerPlates(T exts) 23 T exts ← [T exts \ BoilerP lates] 24 XtremF reqP hrases ← FindFrequentPhrases(T exts ) 25 T ext ← ConflateFreqPhrasesIntoSingleWords(T exts , XtremF reqP hrases) 26 SkipGrams ← ComputeSkipGrams(T exts ) 27 SkipGramsHashes ← HashSkipGrams(SkipGrams) 28 Initialize list ReusedF ragments 29 for i ← 1 to T do 30 for j ← 1 to T do 31 BaseM atches ← GetMatchingHashes(T i , T j , SkipGramHashes) 32 F ullM atches ← ExtendMatches(BaseM atches) 33 Append(ReusedF ragments, F ullM atches) between the 20-gram segments) and the resulting text fragments are marked as boilerplate passages. These text fragments are ignored by the subsequent stages of the approximate-matching algorithm, allowing them to focus on the more meaningful parts of the text, without getting bogged down in these commonly-recurring exact matches.…”

“…Below we discuss specific instantiations of these functions. Our algorithm can be seen as a word-embedding-based variant of the Variability-based Neighbor Clustering (VNC) algorithm for periodization [26], where we replace the measure of variability by a distance measure based on word-embedding matrices. Word embeddings are attractive to use for this purpose because they provide a soft notion of language use, with similar words having similar vectors in the word embedding space [47].…”