Evaluating continuous measurement: Data quality in the Bronx test site of the American community survey

Abstract

The American Community Survey (ACS) is a continuous measurement survey program designed to replace the census long form sample. While the census sample provides detailed socioeconomic data once a decade, the ACS will provide these data annually using a questionnaire that largely mirrors the census long form. This paper examines operational data in Bronx County, one of 36 ACS test counties, and finds the ACS superior to the census on two important measures of data quality. First, applying the minimal data requirement for inclusion of a long form in the census sample, ACS questionnaires were significantly more likely than the long forms to meet this threshold. Second, the level of nonresponse to items on the ACS was often one-half the census level.The ACS and census have sharply divergent goals and operational methods that affect data quality. The primary objective of the census is a population count for reapportionment and redistricting; collecting long form information is a secondary goal. In contrast, the primary objective of the ACS is to estimate the characteristics of an area based on answers to all items on the survey. Operationally, the census follows up every nonresponding household, using a pool of minimally trained, temporary workers. In comparison, the ACS follows up only one-in-three nonresponding households, using a permanent cadre of professional interviewers. This results in superior nonresponse follow-up in the ACS, with lower levels of non-sampling error, even in more difficult-to-enumerate high poverty areas of the Bronx.The ACS is to be fully implemented in July 2004, pending Congressional funding. With detailed socioeconomic data available annually from the ACS, the 2010 census would then be able to focus primarily on a population headcount. The paper concludes by discussing operational aspects of the ACS that must be improved, including increasing its visibility, for the survey to be accepted as an integral part of the nation's data collection system.

Moving from a decennial census to a continuous measurement survey: factors affecting nonresponse at the neighborhood level

“…Questionnaires returned via mail tend to be more complete than those obtained through nonresponse follow-up (Salvo et al, 2003); moreover, the greater the number of questionnaires returned by mail, the fewer the number of households that require nonresponse follow-up. Given its higher mail return rates, the census has a clear initial advantage over the ACS.…”

“…Prior research has shown that in most test sites, the operational methods used in the ACS result in lower levels of item allocation than in the census (Love & Griffin, 2003). Indeed, for the Bronx, despite a mail return that was far lower than that of the census, more efficient nonresponse followup resulted in lower allocation levels, relative to the census (Salvo et al, 2003).…”

“…An index of 2.5 or more is considered by the Census Bureau to constitute an interview; any household that fails to make this threshold is deemed a ''noninterview'' for weighting purposes. Earlier research using the ''sample data-defined'' criterion on both the census and the ACS data for the Bronx found that 22% of all census long forms failed the ''minimal completeness'' test, compared to 11% for the ACS (Salvo, Lobo, & Love, 2003).…”

CEMAF as a Census Method

“…GAO 2003;Wright 1998;Wright and Hogan 2000). In spite of methodological developments such as the de-coupling of the long form from the decennial census (Federal Register 2010;Hough and Swanson 1998Salvo et al 2002;U.S. Census Bureau 2004c, 2010a, nothing has occurred that would suggest to us that methodological developments will reduce litigation and other forms of conflict over census results (see, e.g., Cruickshank 2010; Khavkine 2010; Rowland 2009).…”

The roots of conflict over US census counts in the late 20th century and prospects for the 21st century

“…Perhaps it is not a surprise that the focus by academics, stakeholders, the Census Bureau, and the Congress is largely on methodological developments as the solution to census conflicts -increasing census accuracy through advertising to increase participation, for example, or by using statistical adjustments to reduce differential net undercounts [2,8,15,39,43,[50][51][52][53]. In spite of methodological developments such as the de-coupling of the long form from the decennial census [54][55][56][57], however, nothing has occurred that would suggest to us that methodological developments will reduce litigation and other forms of conflict over census results. In fact, there is evidence to suggest that conflicts over census counts will only intensify, as the realization of net undercount differentials extend beyond the current stakeholders (e.g., political, social, and economic interest groups) to include others -residents of small areas of geography, for example [15,58,59].…”