Summary results of the 2014-2015 DARPA Chikungunya challenge

Abstract

Background: Emerging pathogens such as Zika, chikungunya, Ebola, and dengue viruses are serious threats to national and global health security. Accurate forecasts of emerging epidemics and their severity are critical to minimizing subsequent mortality, morbidity, and economic loss. The recent introduction of chikungunya and Zika virus to the Americas underscores the need for better methods for disease surveillance and forecasting.Methods: To explore the suitability of current approaches to forecasting emerging diseases, the Defense Advanced Research Projects Agency (DARPA) launched the 2014–2015 DARPA Chikungunya Challenge to forecast the number of cases and spread of chikungunya disease in the Americas. Challenge participants (n=38 during final evaluation) provided predictions of chikungunya epidemics across the Americas for a six-month period, from September 1, 2014 to February 16, 2015, to be evaluated by comparison with incidence data reported to the Pan American Health Organization (PAHO). This manuscript presents an overview of the challenge and a summary of the approaches used by the winners.Results: Participant submissions were evaluated by a team of non-competing government subject matter experts based on numerical accuracy and methodology. Although this manuscript does not include in-depth analyses of the results, cursory analyses suggest that simpler models appear to outperform more complex approaches that included, for example, demographic information and transportation dynamics, due to the reporting biases, which can be implicitly captured in statistical models. Mosquito-dynamics, population specific information, and dengue-specific information correlated best with prediction accuracy.Conclusion: We conclude that with careful consideration and understanding of the relative advantages and disadvantages of particular methods, implementation of an effective prediction system is feasible. However, there is a need to improve the quality of the data in order to more accurately predict the course of epidemics.

Local and regional dynamics of chikungunya virus transmission in Colombia: the role of mismatched spatial heterogeneity

“…Faced with these large epidemics, limited resources need to be targeted towards areas with the highest transmission and the most vulnerable populations. In addition, public health officials would like to be able to predict where epidemics of these diseases may spread next [ 12 ].…”

Chikungunya virus outbreak in the Amazon region: replacement of the Asian genotype by an ECSA lineage

“…In this context, it is challenging to use only clinical-epidemiological and serological data to evaluate the true extent of the disease. Moreover, accurate incidence data is critical to forecast and provide prediction of the course of epidemics [22].…”

The future of influenza forecasts

“…In line with other infectious disease challenges (10,12), the study by Reich et al (13) shows that statistical approaches perform particularly well, especially given the short time horizon. It is important to keep in mind that existing prediction targets rely on noisy epidemiological data that are proxies of disease activity (here, weekly ILI incidence), rather than the true accumulation of patients with influenza virus infection, which is unobservable directly.…”

“…In parallel, mechanistic transmission models have benefited from computational advances and extensive data on the mobility and sociodemographic structure of human populations (5,6). In this rapidly advancing research landscape, modeling consortiums have generated systematic model comparisons of the impact of new interventions and ensemble predictions of outbreak trajectory, for use by decision makers (7)(8)(9)(10)(11)(12). Despite the rapid development of disease forecasting as a discipline, however, and the interest of public health policy makers in making better use of analytics tools to control outbreaks, forecasts are rarely operational in the same way that weather forecasts, extreme events, and climate predictions are.…”