The Early Bird Catches The Term: Combining Twitter and News Data For Event Detection and Situational Awareness

Thapen, Nicholas; Simmie, Donal; Hankin, Chris

doi:10.48550/arxiv.1504.02335

Cited by 1 publication

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Freebase online database [1] was used to capture a representative set of symptoms. The exact process is described in [23]. The end product was a list of 46 symptoms, each represented by groups of keywords.…”

Section: Symptom Focusmentioning

confidence: 99%

“…In a previous paper [23] we developed a methodology for identifying areas of high tweet activity within a country or region. This was done by clustering the geo-located tweets using the DBSCAN algorithm.…”

Section: Locationmentioning

confidence: 99%

“…There are multiple advantages to this data-driven generalised location system. Firstly as mentioned in [23] it is transferable to any new area within which there is sufficient tweet density to create clusters. Secondly locations which are not spatial neighbours but exhibit high population movement, for example London and Birmingham, can be considered to be "closer" than they are by distance alone.…”

Section: Locationmentioning

confidence: 99%

“…The node/symptom tweet counts are used as input for both outbreak detection and the clinical disease tracking application. The TNT event summarisation service (see [23]) uses the alarms generated by the outbreak detection algorithm and fetches the source tweets from the processed tweet store with the aim of revealing any potential cause of the symptom event. The news data, which is ingested daily into the system, is also used as input to the event summarisation service.…”

Section: Architecturementioning

confidence: 99%

“…Currently, identifying the causal explanation for a detected event is generally a manual process. Our situational awareness system uses frequency statistics and cosine similarity based measures, outlined in detail in a previous paper by the authors [23], to produce terms characterising the event and then retrieve relevant news and representative tweets. Our generalised symptom focus extends to disease tracking.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

DEFENDER: Detecting and Forecasting Epidemics using Novel Data-analytics for Enhanced Response

Simmie¹,

Thapen²,

Hankin³

2015

Preprint

Self Cite

View full text Add to dashboard Cite

In recent years social and news media have increasingly been used to explain patterns in disease activity and progression. Social media data, principally from the Twitter network, has been shown to correlate well with official disease case counts. This fact has been exploited to provide advance warning of outbreak detection, tracking of disease levels and the ability to predict the likelihood of individuals developing symptoms.In this paper we introduce DEFENDER, a software system that integrates data from social and news media and incorporates algorithms for outbreak detection, situational awareness, syndromic case tracking and forecasting. As part of this system we have developed a technique for creating a location network for any country or region based purely on Twitter data. We also present a disease count tracking approach which leverages counts from multiple symptoms, which was found to improve the tracking of diseases by 37 percent over a model that used only previous case data. Finally we attempt to forecast future levels of symptom activity based on observed user movement on Twitter, finding a moderate gain of 5 percent over a time series forecasting model.

show abstract

Section: Symptom Focusmentioning

confidence: 99%

Section: Locationmentioning

confidence: 99%

Section: Locationmentioning

confidence: 99%

Section: Architecturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations