Comparing Prevalence Estimates From Population-Based Surveys to Inform Surveillance Using Electronic Health Records

Abstract

IntroductionElectronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples.MethodsWe compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013–2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-of-fit measures.ResultsA ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins.ConclusionA TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

Public and Population Health Informatics: The Bridging of Big Data to Benefit Communities

“…These technological drivers have created a unique opportunity to utilize each side's IT infrastructure to boost the aligned aims and outcomes of both groups. For example, a number of studies included in this review depicted that data collected by a large health system or HIE has been used to study the prevalence of a specific chronic disease in a given geography, a task that has been traditionally accomplished by health departments using exhaustive survey methods for public health needs assessment and monitoring purposes 64 65 66 67 . Moreover, data collected by public health departments are a key source of data required to calculate population-level health measures that will eventually be used to determine global budgets for medical care delivery systems and to assess whether they reach their community health targets in order to achieve substantial financial incentives 68 .…”

Evaluating the Representativeness of US Centricity Electronic Medical Records With Reports From the Centers for Disease Control and Prevention: Comparative Study on Office Visits and Cardiometabolic Conditions

“…Statistical equivalence for the pairwise comparisons of proportional distributions, where appropriate, was evaluated using the two one-sided test (TOST) of equivalence [ 55 , 56 ] with a ±2.5, ±5, and ±7.5 percentage point equivalence margins. Using population summary statistics on mean, SD, and total number of office visits, the TOST procedure available in SAS 9.4 (SAS Institute Inc) was employed.…”

Assessing the National Prevalence of HIV Screening in the United States using Electronic Health Record Data

“…Logistic regression was performed to model the effect of age, sex, race, and insurance status on HIV screening. The two one-sided t test (TOST) with +/- 10 point margin was used to assess the equivalence of group prevalence estimates between Explorys and CDC data as recommended by Tatem et al [15]. Statistical significance was set to p < 0.05.…”