Politics and the German language: Testing Orwell’s hypothesis using the Google N-Gram corpus

Why the quantitative analysis of diachronic corpora that does not consider the temporal aspect of time-series can lead to wrong conclusions

“…Plot A replicates the findings of Caruana-Galizia (2015) for the Nazi word Halbjude and the keyword Frieden. At first glance, there seems to be a positive correlation between the time-series of both words as argued by Caruana-Galizia (2015). However, the color pattern reveals that this could be the result of a spurious model: values for later decades (dark shades of gray) strongly influence the apparent relationship.…”

“…Figure 2 presents four plots that all document an apparent linear relationship. To visualize why I believe that the problem described above is also present in the analysis of Caruana-Galizia (2015), the observed values are colored by decade with earlier decades colored in lighter shades of gray and later decades colored in darker shades of gray (as indicated by the color bar at the bottom of the figure). Plot A replicates the findings of Caruana-Galizia (2015) for the Nazi word Halbjude and the keyword Frieden.…”

“…In the next section, I will outline a procedure to account for this problem and show that this procedure strongly affects the results of an analysis, like the one conducted by Caruana-Galizia (2015). …”

“…The correlations are comparable for Rassenschande and Halbjude, but are not identical. To make sure that the analyses presented here are correct, I manually extracted each times series from the Google Books 1-gram corpus and the total 1-gram frequency for the time span 1870-1946 and recalculated the correlations with identical differences between my results and the results presented by Caruana-Galizia (2015). 2 The reason for the difference might be that Caruana-Galizia (2015) calculates the overall token 1-gram frequency on the basis of all words that appear in the corpus for each year.…”

“…This information is available here: http://storage.googleapis.com/books/ngrams/boo ks/googlebooks-ger-all-totalcounts-20090715.txt (last accessed 04/28/2015). 2 In addition, a replication of the results with R using the ngramr package (Carmody, 2014) Caruana-Galizia (2015) have to be challenged, too. However, the autoregressive integrated moving average (ARIMA or ARMAX) models he uses in order to predict the relative frequency of a keyword on the basis of the POLITY2 score (a measure of the level of democracy, the data are available here: http:// www.systemicpeace.org/inscrdata.html, last accessed on 28 April 2015) are quite sophisticated and fitting such a model requires several conceptual decisions regarding the appropriate ARIMA structure that depend on each respective time-series (Becketti, 2013, ch.…”