Use of the Chi-Squared Test to Examine Vocabulary Differences in English Language Corpora Representing Seven Different Countries

Getting rid of the Chi-square and Log-likelihood tests for analysing vocabulary differences between corpora

“…It follows that the presence of some very specific texts, or even a single one, in a corpus may be sufficient to increase the frequency of certain words and thus to modify the words considered as being significantly more frequent in this corpus according to the Chi 2 and LL tests. This phenomenon is perfectly illustrated in the following example reported in Oakes and Farrow (2007) It is important to note that it is not just such extreme cases that invalidate the Chi 2 and LL tests. The simple fact that the probability of a word occurring in a text for a second time is far higher than that of having it for the first time, shows that non-independence is general and not occasional (Church, 2000).…”

“…An advantage of the range over many other measures of dispersion is that it is easily interpretable. It is important to compare the performance of the tests with and without a dispersion threshold because few studies use them, whereas Oakes and Farrow (2007) have shown that it is useful for filtering uninteresting words when using the Chi 2 test. Table 2 summarizes the main results of the analyses.…”

“…A second solution is to use a dispersion measure to eliminate words that only occur in a part of a corpus (Baker, 2004;Oakes & Farrow, 2007). The first problem with this solution is that the threshold used to decide that a word is insufficiently dispersed is necessarily arbitrary, which is all the more annoying since the main measures proposed in the literature are difficult to interpret (Oakes & Farrow, 2007).…”

“…The first problem with this solution is that the threshold used to decide that a word is insufficiently dispersed is necessarily arbitrary, which is all the more annoying since the main measures proposed in the literature are difficult to interpret (Oakes & Farrow, 2007). More-over, Bestgen (2014) showed that taking dispersion into account made it possible to reduce the problem posed by very badly dispersed words (like thalidomide), but not to eliminate it.…”

“…Many studies in corpus linguistics aim to analyse lexical differences between corpora of different genres (Tribble, 2000), their regional and diatypic varieties (Oakes & Farrow, 2007), their oral or written modalities (Rayson, Leech & Hodges, 1997), the period of writing (Laviosa, Pagano, Kemppanen & Ji, 2017) or certain sociological characteristics of the speaker or writer, such as gender, age and socio-economic status (Brezina & Meyerhoff, 2014;Marquilhas, 2015), to cite a few examples. This kind of study immediately raises the question of how to decide whether a difference observed when comparing two given corpora (i.e., more occurrences of towards or male in an American English as opposed to a British English corpus) is purely accidental, or whether it reflects a real difference in the way English is used.…”