Understanding Detection Performance in Public Health Surveillance: Modeling Aberrancy-detection Algorithms

Buckeridge, David L.; Okhmatovskaia, Anna; Tu, Samson W.; O’Connor, Martin J.; Nyulas, Csongor; Musen, Mark A.

doi:10.1197/jamia.m2799

Cited by 31 publications

(25 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If simulated data have often been used to compare statistical approaches to cluster detection [28,[31][32][33][34][35], we could not find prior reports of the use of whole-system simulations to determine how case-detection methods affect outbreak-detection performance. The CDAs evaluated in this study were developed against a validated manual reference standard and only included EMR data elements found to contribute to ARI detection [19].…”

Section: Discussionmentioning

confidence: 99%

Epidemic Surveillance Using an Electronic Medical Record: An Empiric Approach to Performance Improvement

et al. 2014

View full text Add to dashboard Cite

BackgroundsElectronic medical records (EMR) form a rich repository of information that could benefit public health. We asked how structured and free-text narrative EMR data should be combined to improve epidemic surveillance for acute respiratory infections (ARI).MethodsEight previously characterized ARI case detection algorithms (CDA) were applied to historical EMR entries to create authentic time series of daily ARI case counts (background). An epidemic model simulated influenza cases (injection). From the time of the injection, cluster-detection statistics were applied daily on paired background+injection (combined) and background-only time series. This cycle was then repeated with the injection shifted to each week of the evaluation year. We computed: a) the time from injection to the first statistical alarm uniquely found in the combined dataset (Detection Delay); b) how often alarms originated in the background-only dataset (false-alarm rate, or FAR); and c) the number of cases found within these false alarms (Caseload). For each CDA, we plotted the Detection Delay as a function of FAR or Caseload, over a broad range of alarm thresholds.ResultsCDAs that combined text analyses seeking ARI symptoms in clinical notes with provider-assigned diagnostic codes in order to maximize the precision rather than the sensitivity of case-detection lowered Detection Delay at any given FAR or Caseload.ConclusionAn empiric approach can guide the integration of EMR data into case-detection methods that improve both the timeliness and efficiency of epidemic detection.

show abstract

Section: Discussionmentioning

confidence: 99%

Epidemic Surveillance Using an Electronic Medical Record: An Empiric Approach to Performance Improvement

et al. 2014

View full text Add to dashboard Cite

show abstract

“…Automated surveillance data are typically analyzed using aberration detection algorithms 6. Different algorithms have been proposed and evaluated in syndromic surveillance systems 7–9.…”

Section: Introductionmentioning

confidence: 99%

Application of change point analysis to daily influenza-like illness emergency department visits

Kass-Hout

McMurray

et al. 2012

J Am Med Inform Assoc

View full text Add to dashboard Cite

BackgroundThe utility of healthcare utilization data from US emergency departments (EDs) for rapid monitoring of changes in influenza-like illness (ILI) activity was highlighted during the recent influenza A (H1N1) pandemic. Monitoring has tended to rely on detection algorithms, such as the Early Aberration Reporting System (EARS), which are limited in their ability to detect subtle changes and identify disease trends.ObjectiveTo evaluate a complementary approach, change point analysis (CPA), for detecting changes in the incidence of ED visits due to ILI.Methodology and principal findingsData collected through the Distribute project (isdsdistribute.org), which aggregates data on ED visits for ILI from over 50 syndromic surveillance systems operated by state or local public health departments were used. The performance was compared of the cumulative sum (CUSUM) CPA method in combination with EARS and the performance of three CPA methods (CUSUM, structural change model and Bayesian) in detecting change points in daily time-series data from four contiguous US states participating in the Distribute network. Simulation data were generated to assess the impact of autocorrelation inherent in these time-series data on CPA performance. The CUSUM CPA method was robust in detecting change points with respect to autocorrelation in time-series data (coverage rates at 90% when −0.2≤ρ≤0.2 and 80% when −0.5≤ρ≤0.5). During the 2008–9 season, 21 change points were detected and ILI trends increased significantly after 12 of these change points and decreased nine times. In the 2009–10 flu season, we detected 11 change points and ILI trends increased significantly after two of these change points and decreased nine times. Using CPA combined with EARS to analyze automatically daily ED-based ILI data, a significant increase was detected of 3% in ILI on April 27, 2009, followed by multiple anomalies in the ensuing days, suggesting the onset of the H1N1 pandemic in the four contiguous states.Conclusions and significanceAs a complementary approach to EARS and other aberration detection methods, the CPA method can be used as a tool to detect subtle changes in time-series data more effectively and determine the moving direction (ie, up, down, or stable) in ILI trends between change points. The combined use of EARS and CPA might greatly improve the accuracy of outbreak detection in syndromic surveillance systems.

show abstract

“…There is also evidence of ontologies being used in other intricate processes of the software development cycle such as requirements elicitation (Kaiya and Saeki 2006). There are various ontology-based applications that have been developed for the healthcare industry: a syndromic surveillance application to detect disease outbreaks (Buckeridge et al 2008), and extracting meta-data from images to reason and detect early stages of lymphoma in patients (Zillner and Sonntag 2012). These applications are based on the data previously collected in clinics and hospitals.…”

Section: Software Engineering and Ontologiesmentioning

confidence: 99%

An ontology-driven approach to mobile data collection applications for the healthcare industry

Henriques

Lamanna

Kotowski

et al. 2013

Netw Model Anal Health Inform Bioinforma

View full text Add to dashboard Cite

Technology has often been associated with improvements in many domains. This is particularly true in the medical and healthcare industry. This is a field where data collection is performed on a daily basis. With the advent of mobile technology, several methodologies for data collection have been adopted to reduce the cost and time expended on data collection. The focus of this paper is a proposed ontology-based framework that has the ability to build a shared repository of surveys that can be used for data collection. The paper discusses iCollect, a first instantiation of the framework in the form of a survey application built for the Indigenous Health Adaptation to Climate Change (IHACC) project.

show abstract

Understanding Detection Performance in Public Health Surveillance: Modeling Aberrancy-detection Algorithms

Abstract: The validated model of aberrancy-detection algorithms and its software implementation will enable principled comparison of algorithms, synthesis of results from evaluation studies, and identification of surveillance algorithms for use in specific public health settings.

Cited by 31 publications

References 25 publications

Epidemic Surveillance Using an Electronic Medical Record: An Empiric Approach to Performance Improvement

Epidemic Surveillance Using an Electronic Medical Record: An Empiric Approach to Performance Improvement

Application of change point analysis to daily influenza-like illness emergency department visits

An ontology-driven approach to mobile data collection applications for the healthcare industry

Contact Info

Product

Resources

About