A Framework for Evaluation Technical Assistance

Zief, Susan; Knab, Jean; Cole, Russell

doi:10.2105/ajph.2016.303365

Cited by 6 publications

(5 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is a collection of high-quality evaluations, with analyses and results that have been guarded against identifying spurious findings (P-hacking) as a result of prespecified analysis plans and multiple rounds of independent review. 1 Therefore, we can trust these impact results as credible estimates of program effectiveness, and they should become a part of the knowledge base for adolescent pregnancy prevention research.…”

Section: Comprehensive Reporting Of Adolescent Pregnancy Prevention Pmentioning

confidence: 99%

The Teen Pregnancy Prevention Program (2010-2015): Synthesis of Impact Findings

Farb

Margolis

2016

Am J Public Health

View full text Add to dashboard Cite

Section: Comprehensive Reporting Of Adolescent Pregnancy Prevention Pmentioning

confidence: 99%

The Teen Pregnancy Prevention Program (2010-2015): Synthesis of Impact Findings

Farb

Margolis

2016

Am J Public Health

View full text Add to dashboard Cite

“…The process of reviewing questionnaires and applications was very timely, as the first collection of proposals was not as high quality as expected. As a result, NEEP communicated extensively with CSOs to clarify and strengthen their evaluation approach, which was a similar experience to other ETA programs (Holin et al, 2012; Zief et al, 2016). Future ETA programs would benefit from conducting a webinar or virtual meeting with PQQ selectees to clarify the requirements for the RFA to improve the quality of received applications.…”

Section: The Three Tiers Of the Eta Modelmentioning

confidence: 91%

“…Few donors fund rigorous evaluations in development (Merchant-Vega, 2011)—partially due to costs related to conducting impact evaluations, especially if ETA is required. Holin et al (2012) and Zief et al (2016) both noted the resource intensity and costliness of providing effective ETA. Taking into account DFID’s (2011) value for money principles, NEEP was able to utilize smaller grants (US$84,000–US$262,000) to support robust, small-scale impact evaluations (US$107,000–$1.86 million total budgets).…”

Section: Judicious Use Of Funding For the Modelmentioning

confidence: 99%

“…In addition to academic partnerships and individual EETA, NEEP grantees noted their appreciation for the opportunities and platforms to interact with the other CSOs—specifically the global workshops and learning sessions. Other ETA programs have highlighted the use of similar technical workshops (Gibbs et al, 2009; Zief et al, 2016). In addition to developing technical skills at these workshops, grantees built relationships with each other, which led to the sharing of tools, resources, and experiences.…”

Section: Dissemination and Learning For The Modelmentioning

confidence: 99%

See 1 more Smart Citation

Nutrition Embedding Evaluation Programme: An Evaluation Technical Assistance Model for Supporting Civil Society Organizations to Conduct Quality Nutrition Impact Evaluations

et al. 2019

View full text Add to dashboard Cite

The Nutrition Embedding Evaluation Programme (NEEP) was a global 4-year program (2013–2017) funded by the United Kingdom Department for International Development created to respond to gaps in the nutrition evidence base. The NEEP implementing agency—PATH—provided grants and evaluation technical assistance (ETA) to civil society organizations (CSOs) from 12 countries to conduct robust nutrition-related impact evaluations. The programmatic approach of having an intermediary agent to manage the funding and ETA mechanisms for nutrition impact evaluations is rare and therefore provides a unique opportunity to understand its effectiveness. Over the program duration, NEEP collected lessons learned that were analyzed and disaggregated into key themes considered critical for the completion of high-quality impact evaluations. From these lessons learned, NEEP provides an ETA program model that can be replicated or adapted to other international development sectors. This model highlights the key role of the three tiers (donor, ETA manager, and CSOs) in ensuring the best value for money and effective technical support for conducting impact evaluations and fostering the importance of knowledge uptake and evaluative culture for maximum knowledge diffusion. In this way, global research can be targeted to approaches that provide options to collaborate with the program implementers and contribute to a holistic evidence base to inform policy and programmatic decisions.

show abstract

“…The ETA team worked one-on-one with grantees to progress through a series of evaluation milestones that occurred during different stages of the 5-year grant period (see Zief et al, 2016, for details on how this framework worked with Cohort 1 grantees). Early in the grant period, ETA efforts focused on shaping and refining evaluation designs.…”

Section: Eta Frameworkmentioning

confidence: 99%

Teen Pregnancy Prevention Evaluation Technical Assistance: A Case Study

Knab

Cole

2020

Eval Rev

Self Cite

View full text Add to dashboard Cite

Purpose: This case study discusses Mathematica’s experience providing large-scale evaluation technical assistance (ETA) to 65 grantees across two cohorts of Teen Pregnancy Prevention (TPP) Program grants. The grantees were required to conduct rigorous evaluations with specific evaluation benchmarks. This case study provides an overview of the TPP grant program, the evaluation requirements, the ETA provider, and other key stakeholders and the ETA provided to the grantees. Finally, it discusses the successes, challenges, and lessons learned from the effort. Conclusion: One important lesson learned is that there are two related evaluation features, strong counterfactuals and insufficient target sample sizes, that funders should attend to prior to selecting awardees because they are not easy to change through ETA. In addition, if focused on particular outcomes (for TPP, the goal was to improve sexual behavior outcomes), the funder should prioritize studies with an opportunity to observe differences in these outcomes across conditions; several TPP grantees served young populations, and sexual behavior outcomes were not observed or were rare, limiting the opportunity to observe impacts. Unless funders are attentive to weaning out evaluations with critical limitations during the funding process, requiring grantees to conduct impact evaluations supported by ETA might unintentionally foster internally valid, yet underpowered studies that show nonsignificant program impacts. The TPP funder was able to overcome some of the limitations of the grantee evaluations by funding additional evidence-building activities, including federally led evaluations and a large meta-analysis of the effort, as part of a broader learning agenda.

show abstract

A Framework for Evaluation Technical Assistance

Cited by 6 publications

References 2 publications

The Teen Pregnancy Prevention Program (2010-2015): Synthesis of Impact Findings

The Teen Pregnancy Prevention Program (2010-2015): Synthesis of Impact Findings

Nutrition Embedding Evaluation Programme: An Evaluation Technical Assistance Model for Supporting Civil Society Organizations to Conduct Quality Nutrition Impact Evaluations

Teen Pregnancy Prevention Evaluation Technical Assistance: A Case Study

Contact Info

Product

Resources

About