Exploiting the Web for Semantic Change Detection

Abstract

Detecting significant linguistic shifts in the meaning and usage of words has gained more attention over the last few years. Linguistic shifts are especially prevalent on the Internet, where words' meaning can change rapidly. In this work, we describe the construction of a large diachronic corpus that relies on the UK Web Archive and we propose a preliminary analysis of semantic change detection exploiting a particular technique called Temporal Random Indexing. Results of the evaluation are promising and give us important insights for further investigations.

Mining the UK Web Archive for Semantic Change Detection

“…Early work on semantic change detection relied primarily on the comparison of word frequency and co-occurrence patterns between words at different time intervals (Sagi et al, 2009;Cook and Stevenson, 2010;Gulordava and Baroni, 2011), most often representing a single word based on its context (Mihalcea and Nastase, 2012;Jatowt and Duh, 2014;Basile and McGillivray, 2018). Recently, word embeddings have become the common practice for constructing word representations in NLP (Mikolov et al, 2013).…”

“…The UK Web Domain Dataset 1996-2013 (JISC-UK) contains textual information published in UK-based websites over the time period 1996-2013, thus facilitating the task of semantic change detection in a short-term and fine-grained temporal resolution (Basile and McGillivray, 2018).…”

“…The dataset was processed based on previous work by Basile and McGillivray (2018), resulting in over 20TB of textual data. Instead of generating a single vector representation of a word across all years (e.g., by using Temporal Random Indexing (Basile and McGillivray, 2018)), we treated the concatenated content that was published within each year as a single (annual) document D (t) , t ∈ {2000, ..., 2013} 1 .…”

“…The dataset was processed based on previous work by Basile and McGillivray (2018), resulting in over 20TB of textual data. Instead of generating a single vector representation of a word across all years (e.g., by using Temporal Random Indexing (Basile and McGillivray, 2018)), we treated the concatenated content that was published within each year as a single (annual) document D (t) , t ∈ {2000, ..., 2013} 1 . Following most of the past approaches on semantic change (Kim et al, 2014;Hamilton et al, 2016), we generated word representations by training |T + 1| word2vec models m (t) (one per year), using Skip-Gram with Negative Sampling and excluding all words appearing less than 1,000 times within a year.…”

Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection

“…Finally, we measure the level of semantic change of a word by means of the average cosine similarity between the predicted and actual word representations at each time step of the decoder. Model performance is assessed via rank-based metrics (Basile and McGillivray, 2018;Tsakalidis et al, 2019;Shoemark et al, 2019).…”