Evidence for validity of five secondary data sources for enumerating retail food outlets in seven American Indian Communities in North Carolina

Abstract

BackgroundMost studies on the local food environment have used secondary sources to describe the food environment, such as government food registries or commercial listings (e.g., Reference USA). Most of the studies exploring evidence for validity of secondary retail food data have used on-site verification and have not conducted analysis by data source (e.g., sensitivity of Reference USA) or by food outlet type (e.g., sensitivity of Reference USA for convenience stores). Few studies have explored the food environment in American Indian communities. To advance the science on measuring the food environment, we conducted direct, on-site observations of a wide range of food outlets in multiple American Indian communities, without a list guiding the field observations, and then compared our findings to several types of secondary data.MethodsFood outlets located within seven State Designated Tribal Statistical Areas in North Carolina (NC) were gathered from online Yellow Pages, Reference USA, Dun & Bradstreet, local health departments, and the NC Department of Agriculture and Consumer Services. All TIGER/Line 2009 roads (>1,500 miles) were driven in six of the more rural tribal areas and, for the largest tribe, all roads in two of its cities were driven. Sensitivity, positive predictive value, concordance, and kappa statistics were calculated to compare secondary data sources to primary data.Results699 food outlets were identified during primary data collection. Match rate for primary data and secondary data differed by type of food outlet observed, with the highest match rates found for grocery stores (97%), general merchandise stores (96%), and restaurants (91%). Reference USA exhibited almost perfect sensitivity (0.89). Local health department data had substantial sensitivity (0.66) and was almost perfect when focusing only on restaurants (0.91). Positive predictive value was substantial for Reference USA (0.67) and moderate for local health department data (0.49). Evidence for validity was comparatively lower for Dun & Bradstreet, online Yellow Pages, and the NC Department of Agriculture.ConclusionsSecondary data sources both over- and under-represented the food environment; they were particularly problematic for identifying convenience stores and specialty markets. More attention is needed to improve the validity of existing data sources, especially for rural local food environments.

Assessing the validity of commercial and municipal food environment data sets in Vancouver, Canada

“…0·54-0·69 (for all food outlets), similar to results for local health department listings' sensitivity (0·66) and PPV (0·49) in North Carolina, USA (42) , and to a sensitivity estimate (0·66) for city council data in Newcastle, UK (62) . The municipal data sources' PPV scores were lower, however, than those found in Newcastle city council data (PPV = 0·92) (62) and for South Carolina Department of Health and Environmental Control data (PPV = 0·89) (31) .…”

“…Following a surveying protocol adapted from similar research (42) (see online supplementary material, Supplementary File 1), two researchers visited all commercial streets located within an 800 m line-based buffer surrounding schools, a buffer size chosen because it is the distance most frequently examined in research on the community nutrition environment surrounding schools (43) . The researchers identified, photographed and classified all food outlets; a single researcher also identified, photographed and classified any outlets along each residential street included in the sample.…”

“…The present study compared three classes of outlets: (i) limited-service food outlets, restaurants or coffee shops where customers order at a counter and pay before consuming food or beverages; (ii) convenience stores, which included retail stores primarily offering snack foods or beverages, possibly attached to a pharmacy or gas station; and (iii) grocery stores or supermarkets, comprising retail food stores with the departments of a traditional grocer (dairy, bakery, butcher, deli and produce). These three store types were selected because they are the most commonly used store types in the literature on community nutrition environments surrounding schools (43) , and definitions were adapted from previous research (42,49,53) . Outlets were classified following a modification of the flowchart used by Clary and Kestens (28) (included in the online supplementary material, Supplementary File 1).…”

Neighborhood socioeconomic status and food environment: A 20-year longitudinal latent class analysis among CARDIA participants

“…The electronic business record D&B data are widely used in other neighborhood environment research studies and are currently the only option for retrospective longitudinal studies spanning multiple decades. Yet these data are vulnerable to misclassification error including geospatial inaccuracy, missing data, and classification inaccuracy (Bader, Ailshire et al 2010; Fleischhacker, Rodriguez et al 2012; Han, Powell et al 2012). We were unable to retrospectively field validate the historical food environment data from Exam Years 0-15 but other studies provide field validation of the D&B data from 2009(Liese, Colabianchi et al 2010; Powell, Han et al 2011; Rossen, Pollack et al 2012).…”

Field validation of secondary data sources for enumerating retail tobacco outlets in a state without tobacco outlet licensing

“…Sensitivity using ReferenceUSA, however, was 0.61, lower than found in this study. Two studies using ground-truthing found that D&B had moderate sensitivity (Powell et al, 2011, Fleischhacker et al, 2012) and ReferenceUSA had either fair (Lucan et al, 2013), good (Powell et al, 2011) or very good (Gustafson et al, 2012, Fleischhacker et al, 2012) sensitivity. With the exception of Fleischhacker (2012), previous sensitivities for ReferenceUSA are lower than both the probable (0.84) and actual (0.82) tobacco outlet sensitivity found in this study.…”