Event Prediction With Learning Algorithms—A Study of Events Surrounding the Egyptian Revolution of 2011 on the Basis of Micro Blog Data

Abstract

We aim to predict activities of political nature influencing or reflecting societal‐scale behavior and beliefs by applying learning algorithms to Twitter data. This study focuses on capturing domestic events in Egypt from November 2009 to November 2013. To this extent we study un­derlying communication patterns by evaluating content and metadata of 1.3 million tweets through computationally supported classification, with­out targeting specific keywords or users from the Twitter stream. Support Vector Machine (SVM) and Support Distribution Machine (SDM) classification algorithms are applied to detect and predict societal‐scale unrest. Latent Dirichlet Allocation (LDA) is used to create content‐based input patterns for the SVM while the SDM is used to classify sets of features created from meta‐data. The experiments reveal that user cen­tric approaches based on meta‐data outperform methods employing content‐based input despite the use of well established natural language pro­cessing algorithms. The results show that distri­butions over user centric meta information pro­vide an important signal when detecting and pre­dicting events. Applying this approach can assist policymakers and stakeholders in their efforts toward proactive community management.

Do Machines Replicate Humans? Toward a Unified Understanding of Radicalizing Content on the Open Social Web

“…It has been applied in similar detection scenarios, including detection of online pornography (Tang et al, ) and hate speech (Burnap & Williams, ). We follow the standard process of LDA with Gibbs sampling (Blei, ; Boecking, Hall, & Schneider, ; McCallum, ): For each topic k …”

“…Despite the global outlook of jihadists, the lingua franca of groups like ISIS is Arabic, which is poorly covered in the literature. This is important to note for any language‐based automated detection task, as it has been shown that prediction accuracy is best when the locally dominant language is also employed in conjunction with English (Boecking et al, ). Thus, a serious limitation to our approach is the lack of availability of validated Arabic NLP packages.…”

Filtering Tweets for Social Unrest

“…Others use a cascade of text filters to directly extract tweets [1] or Tumblr posts [5] that identify planned unrest events. Another group retrospectively examines the Egyptian revolution of 2011, using Twitter user metadata, including graph features of user networks, to predict major events [6].…”

Following User Pathways

“…In spite of the prevalence and availability of Application Programming Interfaces (APIs), as well as commercial data mining packages, and an almost endless supply of papers and studies that focus on specific platforms [1,2,5,6,11,15,17], cross platform analyses are lacking in the social media analysis and broader computation social science corpus. This is an oversight, in an age where nearly half of social media users have multiple social media profiles [9].…”