Meta-Evaluation: Discovering What Works Best in Welfare Provision

Abstract

Numerous programmes designed to encourage welfare recipients to move into work and off benefit have been evaluated in the United States. Many have randomly assigned potential participants into ‘experimental’ and ‘control’ groups to generate unbiased estimates of the effectiveness of the programmes. The results of the evaluations have been selectively influential in shaping policy developments on both sides of the Atlantic, but a thorough understanding of the diversity of experience has been lacking. Applying meta-analysis techniques to a specially constructed database of evaluations in over 50 US sites, this article reports on the first programme-level, systematic meta-evaluation of welfare-to-work programmes. The results confirm the superiority of approaches that prioritize immediate work over human-capital investment but reveal that caseload characteristics and local environment can be equally important as or even more important than programme design. The article concludes with a discussion of the potential and limitations of meta-evaluation.

Improving the quality of evaluation participation: a meta-evaluation

“…Meta-evaluations can also aid HRD evaluators in improving their own work and its usefulness. Rather than imposing some external structure such as the Program Evaluation Standards (Joint Committee on Standards for Educational Evaluation 1994) or using meta-analytic methods to combine results from several previous evaluations (for example, Ashworth et al 2004), the HRD researcher or practitioner can focus the meta-evaluative work on specific issues related to the original Human Resource Development International 47 evaluation as guided by the stakeholders for the meta-evaluation. Such a focus fits within the systems model for evaluation (Russ-Eft andPreskill 2001, 2005;Russ-Eft 2005, 2006), which should lead to greater stakeholder involvement and increased evaluation use.…”

Predicting Material Hardship Among Former Welfare-to-Work Participants: An Income- and Resource-Packaging Model

“…It emphasizes immediate job search and employment because this is considered to be the best way to decrease welfare costs and reduce welfare rolls, while allowing participants to build skills that lead to better jobs (Gueron & Hamilton, 2002). Research investigating which program aspects, particularly work activities, are associated with higher earnings has suggested that engaging in unsubsidized employment resulted in increased earnings and/or likelihood of employment (Ashworth, Cebulla, Greenberg, & Walker, 2004;Greenberg, Ashworth, Cebulla, & Walker, 2005;Greenberg, Cebulla, & Bouchet, 2005;Greenberg, Michaloupoulus, & Walker, 2003). Consequently, participation in unsubsidized employment (i.e., work-first participation) is hypothesized to predict a decreased likelihood of experiencing material hardship.…”

A Fresh Look at an Old Debate: Assigned Work Activities, Employment, and Post-program Earnings in TANF Work Programs

“…Recently, scholars have begun using meta-analysis to determine which program aspects, particularly work activities, are associated with higher earnings while controlling for recipients' characteristics and the local socioeconomic environment (Ashworth et al, 2004;Greenberg et al 2005;Greenberg et al, 2003 In general, they find job search has the biggest positive effect on earnings with mixed results for basic education and vocational education training, leading researchers to discourage additional investments in education and training. They also stress the importance of considering local environment and population characteristics.…”

“…The preponderance of the evidence indicates that a work-first strategy produces the best results (Ashworth, Cebulla, Greenberg, & Walker, 2004;Greenberg, Ashworth, Cebulla, & Walker, 2005;Greenberg, Cebulla, & Bouchet, 2005;Greenberg, Michaloupoulus, & Walker, 2003). This article argues that conclusions reached under AFDC warrant additional scrutiny under TANF due to the changing policy environment.…”