A Meta-Analysis of State-of-the-Art Electoral Prediction From Twitter Data

Abstract

Electoral prediction from Twitter data is an appealing research topic. It seems relatively straightforward and the prevailing view is overly optimistic. This is problematic because while simple approaches are assumed to be good enough, core problems are not addressed. Thus, this paper aims to (1) provide a balanced and critical review of the state of the art; (2) cast light on the presume predictive power of Twitter data; and (3) depict a roadmap to push forward the field. Hence, a scheme to characterize Twitter prediction methods is proposed. It covers every aspect from data collection to performance evaluation, through data processing and vote inference.Using that scheme, prior research is analyzed and organized to explain the main approaches taken up to date but also their weaknesses. This is the first meta-analysis of the whole body of research regarding electoral prediction from Twitter data. It reveals that its presumed predictive power regarding electoral prediction has been somewhat exaggerated: although social media may provide a glimpse on electoral outcomes current research does not provide strong evidence to support it can currently replace traditional polls. Finally, future lines of work are suggested.

The empiricist’s challenge: Asking meaningful questions in political science in the age of big data

“…On the one hand, this makes these studies highly popular in that they seemingly offer a fairly straightforward way to measure and predict social, economic, and political phenomena. On the other hand, these studies have been found to be highly vulnerable to replication efforts, indicating that early hopes might be a case of collective overexcitement rather than the hoped-for replacement for more traditional measurement approaches or the prediction of future developments (Gayo-Avello, 2013;Jungherr, Jürgens, & Schoen, 2012;Lazer, Kennedy, King, & Vespignani, 2014;Metaxas, Mustafaraj, & Gayo-Avello, 2011). Here, it is of paramount importance that social science loses its fascination with the proof-of-concept publication model imported from computer science and instead demands sophisticated tests of indicator validation and the theorizing and testing of links between variables ostensibly linked.…”

Political relevance in the eye of the beholder: Determining the substantiveness of TV shows and political debates with Twitter data

“…This is indeed true if one makes inferences about the amount of support for a certain specific policy or candidate, and this is why attempts to use Twitter analysis as a replacement for polling have been extensively criticized (Gayo-Avello, 2013;Jungherr, et al, 2011). Yet, this is less of a concern in our case as we are not interested in obtaining exact estimates of percentages of the population that hold a given opinion.…”

Predicting TV programme audience by using twitter based metrics

“…Voting results have been correlated with tweets in the 2009 German elections [53], addressing the counting of the tweets citing the different parties without providing a predictive model, another example can be found in [4]. In [15], sentiment analysis and volume approaches has been used for electoral prediction in the Senate competition which is 1:1, still obtaining correlations in the range of 40-60%.…”