Information-based Modeling of Diachronic Linguistic Change: from Typicality to Productivity

Abstract

We present a new approach for modeling diachronic linguistic change in grammatical usage. We illustrate the approach on English scientific writing in Late Modern English, focusing on grammatical patterns that are potentially indicative of shifts in register, genre and/or style. Commonly, diachronic change is characterized by the relative frequency of typical linguistic features over time. However, to fully capture changing linguistic usage, feature productivity needs to be taken into account as well. We introduce a data-driven approach for systematically detecting typical features and assessing their productivity over time, using information-theoretic measures of entropy and surprisal.

Measuring diachronic language distance using perplexity: Application to English, Portuguese, and Spanish

“…For this we will take into account the work of Pechenick et al (2015) which studies how to validate corpora for analysis of cultural and linguistic evolution, the research performed by Bochkarev, Solovyev, and Wichmann 2014 where KLD is applied to Google Books Corpus to compare historically the change in the frequency distribution of words within one language and across languages. Also, Degaetano-Ortlieb and Teich (2018) measure the diachronic change at the lexical and grammatical level in scientific writing and Barron et al . (2018) apply KLD to investigate how ideas evolve in Parliamentary transcripts of the French Revolution Corpus.…”

“…To define historical periods for diachronic corpora: Attending to Klarer (2013): “The convention of periodical classification must not distract from the fact that such criteria are relative and that any attempt to relate divergent texts ‘with regard to their structure, contents, or date of publication’ to a single period of literary history is always problematic.” These periods of linguistic change and lexical and grammatical features contributing to change could be detected automatically for each language using the method of Degaetano-Ortlieb and Teich (2018) or the method of identification of stages done by Th. Gries and Hilpert (2008).…”

“…Recently, Degaetano-Ortlieb et al (2016) have presented an information-theoretic approach based on entropy to investigate diachronic change in scientific English, Rama et al (2015) have used cross-entropy to measure distances, and Degaetano-Ortlieb and Teich (2018) have used relative entropy for detection and analysis of periods of diachronic linguistic change.…”

Linguistic Variation and Change in 250 Years of English Scientific Writing: A Data-Driven Approach

“…While well-suited to capture the summative effects of change (groups of words or whole vocabularies, see e.g., Grieve et al, 2016 ), the primary focus lies on lexis 3 . Other linguistic levels, e.g., grammar (Degaetano-Ortlieb and Teich, 2016 , 2018 ; Bizzoni et al, 2019a ), collocations (Xu and Kemp, 2015 ; Garcia and Garćia-Salido, 2019 ), or specific aspects of change, e.g., spread of change (Eisenstein et al, 2014 ), specialization (Bizzoni et al, 2019b ) or life-cycles of language varieties (Danescu-Niculescu-Mizil et al, 2013 ), are only rarely considered. Once again, while word embeddings offer a specific model of language use, using them to capture diachronic change and to assess effects of change calls for adequate instruments for comparison along the time line.…”

Diachronic Variation of Temporal Expressions in Scientific Writing Through the Lens of Relative Entropy

“…5). The methodology has been adopted from Fankhauser et al (2016) and successfully used in Degaetano-Ortlieb and Teich (2016) to detect typical linguistic features in scientific writing, Degaetano-Ortlieb and Teich (2017) to detect typical features of research article sections, and Degaetano-Ortlieb (2017) to observe typical features of social variables.…”