Mining the Web for New Words: Semi-Automatic Neologism Identification with the NeoCrawler

Abstract

Lexical innovation is omnipresent and constantly at work. Studies aiming to understand the process of lexical innovation and the subsequent diffusion of neologisms therefore benefit from systematic methods of neologism identification. Retrieval procedures in the past have largely consisted of manual activities of participant observations and close reading. Recently, attempts have been made at designing automatized identification procedures, assisted by state-of-the-art natural language processing techniques and tools. Beginning with a discussion of the most commonly used neologism detection methods and applications in linguistics, the present paper will describe a semi-automatic approach to identifying new words on the web, the NeoCrawler’s Discoverer, which has been developed as part of a project on the incipient diffusion of lexical innovations. The Discoverer daily processes large batches of online text in English and automatically identifies unknown grapheme sequences as potential neologism candidates by means of a dictionary matching procedure, in which the individual tokens are matched against a very large dictionary. These potential neologisms subsequently are presented to the user for manual evaluation of their neologism status. Finally, candidates are added to the NeoCrawler’s database for continuous close monitoring of their development in the online speech community. We argue that the use of dictionary matching in neologism identification offers an efficient method to semi-automatically extract potential instances of lexical innovation with high precision and high recall when compared to previous approaches.

Num-Symbolic Homophonic Social Net-Words

“…Reference [43] used a subject-based search method for related information, while we used user-generated text collection in a subject-free manner. Reference [44] collected a large number of English online texts routinely and collected candidate words via dictionary matching. Their method was similar to the concept in this research; however, the Chinese language has no natural separators, which makes text analysis more challenging.…”

Detecting and categorising lexical innovations in a corpus of tweets

“…To do so using a Reddit corpus, they used the frequency of words over time to identify the words with increasing frequency using the Spearman coefficient and the words with decreasing frequency by fitting the frequency series of words with a two-phase piecewise linear regression and with a logistic distribution for the more discrete growth and decay trajectories. On the other hand, Kerremans and Prokić (2018), chose a semi-automatic detection of neologisms on the web using correspondence dictionaries.…”

Coming to Terms: Automatic Formation of Neologisms in Hebrew

“…Computational neologism. Much previous computational work on neologisms focused on automatic recognition of neologisms and their meanings (Cook and Stevenson, 2010;Cartier, 2017;Costin-Gabriel and Rebedea, 2014;Veale and Butnariu, 2010;Kerremans and Prokić, 2018). Work on computational generation of neologisms mostly focused on creating compounds and word blends from source words (Smith et al, 2014;Deri and Knight, 2015;Gangal et al, 2017;Kulkarni and Wang, 2018;Özbal and Strapparava, 2012;Simon, 2018).…”