Evaluation and comparison of statistical methods for early temporal detection of outbreaks: A simulation-based study

Abstract

The objective of this paper is to evaluate a panel of statistical algorithms for temporal outbreak detection. Based on a large dataset of simulated weekly surveillance time series, we performed a systematic assessment of 21 statistical algorithms, 19 implemented in the R package and two other methods. We estimated false positive rate (FPR), probability of detection (POD), probability of detection during the first week, sensitivity, specificity, negative and positive predictive values and F1-measure for each detection method. Then, to identify the factors associated with these performance measures, we ran multivariate Poisson regression models adjusted for the characteristics of the simulated time series (trend, seasonality, dispersion, outbreak sizes, etc.). The FPR ranged from 0.7% to 59.9% and the POD from 43.3% to 88.7%. Some methods had a very high specificity, up to 99.4%, but a low sensitivity. Methods with a high sensitivity (up to 79.5%) had a low specificity. All methods had a high negative predictive value, over 94%, while positive predictive values ranged from 6.5% to 68.4%. Multivariate Poisson regression models showed that performance measures were strongly influenced by the characteristics of time series. Past or current outbreak size and duration strongly influenced detection performances.

A Simulation-Based Study on the Comparison of Statistical and Time Series Forecasting Methods for Early Detection of Infectious Disease Outbreaks

“…In their study, the CUSUM method with adaptive regression residuals showed better results than EARS. Recently, Gabriel et al [ 11 ] compared 21 statistical algorithms for temporal outbreak detection. However, these studies did not include time series forecasting methods.…”

Lean back and wait for the alarm? Testing an automated alarm system for nosocomial outbreaks to provide support for infection control professionals

“…Moreover, regression models like Farrington do not perform well for small outbreaks (low number of patients) like those in our hospital and in most other hospital settings [25]. Previously it was shown in a simulation study that Farrington has a low sensitivity [26]. Therefore, it is possible that the Farrington algorithm alone should not be the algorithm of choice for regular hospital outbreaks.…”

Timely detection of Pertussis outbreaks in Iran: The comparison performance of Wavelet-based outbreak detector, Exponential weighted moving average, and Poisson regression-based methods

“…As a result of an increase in the amount of K, the probability of generation alarm on the first day of outbreaks decreased due to an increase in the level of alarm thresholds. The median, minimum and maximum timeliness (according to day) in Haar (k; 0.5) and db10 (k: 0.5) wavelets based method was 2 (1 to 14) and 2 (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14) respectively. This amount was less than the median timeliness of other algorithms.…”

“…collection, analysis, interpretation and final dissemination in public health practice. No doubt, the surveillance system will contribute to reducing the morbidity and mortality due to health-related events (3). Additionally, utilizing a suitable method in a surveillance system for early detection of naturally occurring or bioterrorism-related outbreaks have a very important role in reducing the time between outbreak occurrence and detection (4).…”