Monitoring events with application to syndromic surveillance using social media data

Abstract

Availability of time series data in different domains has resulted in approaches for outbreak detection. A popular alternative to detect outbreaks is to use daily counts of events. However, time between events (TBE) has proven to be a useful alternative, especially in the case of sudden, unexpected events. Past work that uses TBE for monitoring events assumes that the in-control number of events is up to 10 per day. In this article, we derive robust monitoring plans that are scalable when the in-control counts are higher than 10 per day but less than 100 per counting period (eg, day). TBE values are generally nonhomogeneous across days and within days. This makes the volume of data to train the technology a challenge, and this challenge increases the volume of data needed to design the charts. This article discusses these challenges and suggests solutions for data that are known to be Weibull-distributed. We present our results in two parts.The first is a simulated dataset that controls parameters of the plan such as the daily counts of events. We then show how the monitoring plans can be applied to the detection of syndromes (ie, disease outbreaks) using social media data.

Syndromic surveillance to detect disease outbreaks using time between emergency department presentations

“…11-15 However, recently it has been demonstrated that for early detection of disease outbreaks, it is much more efficient to monitor the time between events (TBEs) than to use daily counts. 16,17 The efficiency gain with monitoring TBE is that every individual instance (ED presentation) offers a decision point about a disease outbreak, and we do not have to wait for the end of the day or week to count the overall disease counts to make that decision. When the TBE values for a particular disease get smaller and smaller, then the likelihood of an unusual outbreak for that disease increases.…”

“…TBE are computed for each hospital for influenza-like illness ED presentations and EWMAs of the TBE 16,17 with different levels of temporal memory are calculated. These levels of memory relate to adopting different model coefficients resulting in differences in the ability of the model to correctly signal an outbreak and can be optimised depending on whether daily counts of the monitored signal are low (e.g.…”

“…However, in the context of providing alerts as quickly as possible in continuous surveillance programmes, monitoring the time between events has been shown to be more responsive than using daily counts as we do not have to wait for the end of the day or week to make a decision about an outbreak. 16,17 The TBE approach to statistical process control is a very new concept, and we are one of the few groups working on its application to nonhomogeneous processes worldwide. The monitoring plans use an adaptation of EWMAs, which is a process control method with prior use in detecting anomalies typically for industrial production processes.…”

“…We now know how to design these approaches to be robust at detecting any outbreak. 17 In the present study, outbreak detection models were developed based on presentations of patients with influenza-like illness as coronavirus counts at the study sites decreased after an initial first wave. The approach is applicable to other hospitals but calculation of the test statistic that is compared to the threshold requires TBE >0.…”

A Real Time Monitoring Approach for Bivariate Event Data

“…With a TBE control chart, we can monitor the length of time between events. For recent studies on univariate TBE control charts, the reader is referred to Sparks et al (2019Sparks et al ( , 2020. Methods for multivariate TBE data are categorized into two types; methods for (a) vector-based event data and (b) pointprocess data (Zwetsloot et al, 2021).…”