Lessons Learned from the First Year of Implementation of the Centers for Disease Control and Prevention's Standardized Evaluation System for HIV Prevention Programs

Abstract

In December 1999 the Centers for Disease Control and Prevention's (CDC's) Division of HIV/AIDS Prevention initiated a standardized evaluation system for CDC-funded health department HIV prevention programs. This health department evaluation guidance asks health departments to develop comprehensive evaluation plans and to submit aggregated data on such activities as intervention planning, process monitoring, and outcome evaluation. During the first year of this system, of 65 health departments, 62 submitted evaluation plans, 37 submitted intervention plan data, and 20 submitted process monitoring data. Major issues affecting implementation of a national evaluation system include varying levels of evaluation capacity among health departments, differences between the CDC's taxonomy for national data collection and local definitions, and limitations regarding use of 1st-year data. The CDC has learned that implementation of a standardized evaluation system takes considerable time and that stakeholder involvement and technical assistance and capacity building support are essential.

Assessing Training Needs of HIV Program Providers

“…Each year the CDC funds prevention programs. For example, Glassman, Lacson, Collins, Hill, and Wan (2002) reported that in fiscal year 2000 the CDC awarded nearly $287 million in cooperative agreement funding to 65 health departments for HIV-prevention programs. They also noted that the recipients, in turn, negotiated grants to communitybased organizations (CBOs) to implement programs for targeted risk populations.…”

“…Evaluation is increasingly viewed as a way to improve the effectiveness and accountability of prevention programs (Gilliam, Davis, Barrington, Lacson, & Uhl, 2002;Glassman, Lacson, Collins, Hill, & Wan, 2002). Federal, state, and local decision makers need accurate estimates of the effectiveness and the costs of various HIV prevention efforts so they can wisely allocate limited resources (Pinkerton, Chesson, Holtgrave, Kassler, & Layde, 2000).…”

“…The CDC now asks health departments to submit aggregated outcome monitoring data. However, as Glassman et al (2002) noted, valid and reliable aggregated data requires valid and reliable local data. Local prevention programs face many barriers in obtaining such data.…”

Audit Culture

“…For HIV prevention, each year the Centers for Disease Control and Prevention (CDC), state and local health departments, and philanthropic organizations spend hundreds of millions of dollars to support HIV prevention programming. Therefore, program evaluation is critical for accountability, program planning, and quality assessment (Glassman, Lacson, Collins, Hill, & Wan, 2002). As others have argued (Chouinard, 2013a, 2013b), the standard approach to evaluation remains the collection of impartial, evidence-based, and objective information in the form of quantifiable measurements in order to satisfy accountability requirements.…”

“…Funders' continued support for these programs is often contingent upon agencies documenting that funds are being used efficiently and program recruitment goals are being met. For agencies implementing DEBI interventions and funded by diverse entities, program impact and process monitoring focus on recording and reporting various numerical data, including number of clients to be reached (by race, ethnicity, gender, and age), number of clients served, and number of clients who receive each intervention session (Chen, 2001;Dodd & Meezan, 2003;Glassman et al, 2002). In this article, we use the lens of ''audit culture'' to understand the consequences of numbers-based evaluation and accountability to funders on frontline service providers.…”