Tracking Dengue Epidemics Using Twitter Content Classification and Topic Modelling

Abstract

Abstract. Detecting and preventing outbreaks of mosquito-borne diseases such as Dengue and Zika in Brasil and other tropical regions has long been a priority for governments in affected areas. Streaming social media content, such as Twitter, is increasingly being used for health vigilance applications such as flu detection. However, previous work has not addressed the complexity of drastic seasonal changes on Twitter content across multiple epidemic outbreaks. In order to address this gap, this paper contrasts two complementary approaches to detecting Twitter content that is relevant for Dengue outbreak detection, namely supervised classification and unsupervised clustering using topic modelling. Each approach has benefits and shortcomings. Our classifier achieves a prediction accuracy of about 80% based on a small training set of about 1,000 instances, but the need for manual annotation makes it hard to track seasonal changes in the nature of the epidemics, such as the emergence of new types of virus in certain geographical locations. In contrast, LDA-based topic modelling scales well, generating cohesive and well-separated clusters from larger samples. While clusters can be easily re-generated following changes in epidemics, however, this approach makes it hard to clearly segregate relevant tweets into well-defined clusters.

Recruiting from the Network: Discovering Twitter Users Who Can Help Combat Zika Epidemics

“…In our previous work [10], we used topic modelling similar to that shown in [15], however, we focused on pre-defined classes of interest specifically related to Zika epidemics. They use community detection (Louvain modularity) and the encoding of random walks to detect community structures within the topics they previously defined.…”

“…Filtering keywords were selected in two steps, following an approach similar to that suggested in [12]. Firstly, a short list of seed keywords was bootstrapped from sample tweets content using manual, expert inspection, and borrowing from our earlier work [10]. These are the top 8 keywords: dengue, combateadengue, focodengue, todoscontradengue, aedeseagypti, zika, chikungunya, virus.…”

“…We aimed to learn a classifier that effectively provides an operational definition of relevance of tweets in the context of our topic. In our previous work on detecting Dengue-related tweets [10] we used four target classes with the same purpose: Mosquito-focus, Sickness, News, and Joke, representing (i) content that is strictly relevant to the topic, (ii) content that describes symptoms by affected people, (iii) content from news agencies or that echoes news from agencies, and (iv) content from people who make mostly sarcastic or humorous remarks about Dengue, respectively. In that setting, both Mosquito-focus and Sickness tweets would be considered relevant.…”

“…Our experimental dataset consists of a harvest of 278,351 tweets, collected and classified through our online pipeline (Fig.1) using the keywords presented in Sec. 2.1 during a period of 4 months (9)(10)(11)(12) in 2016. Using our classifier, we found 15,124 Relevant tweets in this set.…”

Analysing Cultural Events on Twitter

“…and that it led to a lesser engagement and a low audience participation. As mentioned in [7], tweet analysis has led to a large number of studies in many do-mains such as ideology prediction in Information Sciences [4], natural disaster anticipation in Emergency [15] and tracking epidemic [13] while work in Social Sciences and Digital Humanities has developed tweet classifications [16]. However, few studies aim at classifying tweets according to communication classes.…”