Predicting Social Unrest Events with Hidden Markov Models Using GDELT

Abstract

Proactive handling of social unrest events which are common happenings in both democracies and authoritarian regimes requires that the risk of upcoming social unrest event is continuously assessed. Most existing approaches comparatively pay little attention to considering the event development stages. In this paper, we use autocoded events dataset GDELT (Global Data on Events, Location, and Tone) to build a Hidden Markov Models (HMMs) based framework to predict indicators associated with country instability. The framework utilizes the temporal burst patterns in GDELT event streams to uncover the underlying event development mechanics and formulates the social unrest event prediction as a sequence classification problem based on Bayes decision. Extensive experiments with data from five countries in Southeast Asia demonstrate the effectiveness of this framework, which outperforms the logistic regression method by 7% to 27% and the baseline method 34% to 62% for various countries.

iCoRe: The GDELT Interface for the Advancement of Communication Research

“…As our case-studies and other research drawing on GDELT has demonstrated (e.g., Qiao et al, 2017;Vargo, et al, 2018), GDELT does provide meaningful, valid insights into larger societal trends and news coverage patterns. Second, communication researchers using iCoRe to obtain computationally derived content-analytic measures must be conscious of the opportunities and shortcomings that underlie these methods .…”

“…In light of GDELT's massive datasets, GDELT's data has been integrated into Google BigQuery, a web service designed to store and provide access to large-scale datasets through standard SQL queries. Accordingly, the majority of studies accessing GDELT have utilized Google BigQuery (e.g., Qiao et al, 2017) or developed independent scripts that download and parse the raw GDELT data in comma-separated value (CSV) format to address a specific research question (e.g., . While Google BigQuery provides unprecedented querying speed, it is a fee-based service that can quickly become expensive with increasingly data-heavy operations.…”

“…GDELT is an open-source, large-scale repository of global online news and events metadata collected from diverse media formats and updated in near real-time. GDELT has sparked considerable interest within cognate social science disciplines, for example, to forecast and understand the latent structure of political unrest and violence in East Asia (Qiao et al, 2017) and the Middle East (Smith, Smith, Legg & Francis, 2017). Despite its clear utility for addressing these critical questions, GDELT has been used by only a handful of communication scholars for conducting theory-driven research.…”

An Online Framework for Civil Unrest Prediction Using Tweet Stream Based on Tweet Weight and Event Diffusion

“…Some articles criticized this method of content classification as keywords alone cannot decide the relativity of content to civil unrest. Instead, they used keyword dictionary for filtering and the popular supervised classifiers like SVM (Chen & Neill, 2014;Korolov et al, 2016;Qiao & Wang, 2015), Naïve Bayes (Qiao et al, 2017;Ramakrishnan et al, 2014;van Noord, Kunneman, & van den Bosch, 2016) and n-gram classifier (Compton et al, 2014) for classification of content. An exception, Ranganath et al (2016) used the latent discriminant classifier based on extracted latent dimension representation of content to identify related content.…”

“…However, these frameworks suffer from the fact that nowadays protestors are smart and rarely specify dates of their planned events directly in their posts. In view of this limitation, other authors use regression techniques like logistics and Least Absolute Shrinkage and Selection Operator (LASSO) regression (Cadena et al, 2015;Korkmaz et al, 2015;Korolov et al, 2016;Qiao & Wang, 2015;Ramakrishnan et al, 2014;Wu & Gerber, 2018), HMM (Hidden Markov Model) (Qiao et al, 2017), random forest (Singh & Pal, 2018) and Long Short-Term Memory Networks (LSTM) (Galla & Burke, 2018) in predicting tasks that use historical event information. The rest of the authors introduce their own prediction models like data mining frameworks based indication and warning assessment, recognition system (IWARS) (Benkhelifa, Rowe, Kinmond, Adedugbe, & Welsh, 2014), graph-based non-parametric heterogeneous graph scan (NPHGS) model (Chen & Neill, 2014) and multi-task learning framework based Multi-Task Feature Learning (MTFL) model (Zhao et al, 2017), Bayesian model fusion framework (Hoegh, Leman, Saraf, & Ramakrishnan, 2015), Naïve Bayes (Hossny & Mitchell, 2018), and keyword frequency-based model (Manrique et al, 2013).…”