Outta Control: Laws of Semantic Change and Inherent Biases in Word Representation Models

Abstract

This article evaluates three proposed laws of semantic change. Our claim is that in order to validate a putative law of semantic change, the effect should be observed in the genuine condition but absent or reduced in a suitably matched control condition, in which no change can possibly have taken place. Our analysis shows that the effects reported in recent literature must be substantially revised: (i) the proposed negative correlation between meaning change and word frequency is shown to be largely an artefact of the models of word representation used; (ii) the proposed negative correlation between meaning change and prototypicality is shown to be much weaker than what has been claimed in prior art; and (iii) the proposed positive correlation between meaning change and polysemy is largely an artefact of word frequency. These empirical observations are corroborated by analytical proofs that show that count representations introduce an inherent dependence on word frequency, and thus word frequency cannot be evaluated as an independent factor with these representations.

Tracing Antisemitic Language Through Diachronic Embedding Projections: France 1789-1914

“…The law of prototipicality was introduced by Du-bossarsky et al (2015): it states that prototypical words, words that are near to the centroid of a cluster in a semantic space, change slower than words that are in a peripheral position. The laws of conformity, innovation and prototipicality have been questioned by Dubossarsky et al (2017), who used controlled conditions to test them.…”

Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change

“…The successful outcome of semantic change detection is relevant to any diachronic textual analysis, including machine translation or normalization of historical texts (Tjong Kim Sang et al, 2017), the detection of cultural semantic shifts (Kutuzov et al, 2017) or applications in digital humanities (Tahmasebi and Risse, 2017a). However, currently, the best-performing models (Hamilton et al, 2016b;Kulkarni et al, 2015;Schlechtweg et al, 2019) require a complex alignment procedure and have been shown to suffer from biases (Dubossarsky et al, 2017). This exposes them to various sources of noise influencing their predictions; a fact which has long gone unnoticed because of the lack of standard evaluation procedures in the field.…”

Empirical Study of DiachronicWord Embeddings for Scarce Data

“…Another broadly used method it to learn an embedding matrix for each time slice independently; due to the stochastic aspect of word embeddings, the vectorial space for each time slice is different, making them not directly comparable. Thus, authors perform an alignment of the embeddings spaces by optimising a geometric transformation (Hamilton et al, 2016;Dubossarsky et al, 2017;Szymanski, 2017;Kulkarni et al, 2015)).…”