Timely evaluation in international development

Webster, Jayne; Exley, Josephine; Copestake, James; Davies, Rick; Hargreaves, James

doi:10.1080/19439342.2018.1543345

Cited by 6 publications

(2 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Intervention evaluation is an essential component of public health research and programming, and detailed guidance on evaluation approaches and methodologies exist 5, 6 . However, the iterative and flexible nature of HCD potentially poses some unique challenges for evaluation 1, 3, 7 and examples of evaluations of HCD public health interventions have so far been limited 7, 8 .…”

Section: Rationalementioning

confidence: 99%

Challenges and opportunities in evaluating programmes incorporating human-centred design: lessons learnt from the evaluation of Adolescents 360

et al. 2019

View full text Add to dashboard Cite

Adolescents 360 (A360) is a four-year initiative (2016–2020) to increase 15-19-year-old girls’ use of modern contraception in Nigeria, Ethiopia and Tanzania. The innovative A360 approach is led by human-centred design (HCD), combined with social marketing, developmental neuroscience, public health, sociocultural anthropology and youth engagement ‘lenses’, and aims to create context-specific, youth-driven solutions that respond to the needs of adolescent girls. The A360 external evaluation includes a process evaluation, quasi-experimental outcome evaluation, and a cost-effectiveness study. We reflect on evaluation opportunities and challenges associated with measuring the application and impact of this novel HCD-led design approach. For the process evaluation, participant observations were key to capturing the depth of the fast-paced, highly-iterative HCD process, and to understand decision-making within the design process. The evaluation team had to be flexible and align closely with the work plan of the implementers. The HCD process meant that key information such as intervention components, settings, and eligible populations were unclear and changed over outcome evaluation and cost-effectiveness protocol development. This resulted in a more time-consuming and resource-intensive study design process. As much time and resources went into the creation of a new design approach, separating one-off “creation” costs versus those costs associated with actually implementing the programme was challenging. Opportunities included the potential to inform programmatic decision-making in real-time to ensure that interventions adequately met the contextualized needs in targeted areas. Robust evaluation of interventions designed using HCD, a promising and increasingly popular approach, is warranted yet challenging. Future HCD-based initiatives should consider a phased evaluation, focusing initially on programme theory refinement and process evaluation, and then, when the intervention program details are clearer, following with outcome evaluation and cost-effectiveness analysis. A phased approach would delay the availability of evaluation findings but would allow for a more appropriate and tailored evaluation design.

show abstract

Section: Rationalementioning

confidence: 99%

Challenges and opportunities in evaluating programmes incorporating human-centred design: lessons learnt from the evaluation of Adolescents 360

et al. 2019

View full text Add to dashboard Cite

show abstract

“…This situation suits adaptive or iterative learning, whereby decisions are informed by direct experience, and good practice is shared within and across organisations and cultures, through coaching, mentoring or demonstration (Webster et al, 2018). Practitioners, particularly in humanitarian settings, often make decisions based on their understanding of the context, rather than relying on formal evaluation and reporting which they see as slow and having little to offer practitioners on the ground.…”

Section: Engaging With Context-specific Evidence To Make Decisionsmentioning

confidence: 99%

Engaging stakeholders with evidence and uncertainty: developing a toolkit

Oliver¹,

Langer²,

Nduku³

et al. 2021

View full text Add to dashboard Cite

CEDIL’s Methods Working Paper 4 ‘Engaging Stakeholders with Evidence and Uncertainty: Developing a Toolkit’, offers a new framework that helps choose appropriate stakeholder engagement methods while conducting research and supporting decision-making. The framework provides the foundation for a toolkit that distinguishes major differences in stakeholder engagement, illustrates pathways for choosing appropriate methods for stakeholder engagement, signposts evidence and practical tools to support stakeholder engagement, and guidance for identifying and understanding stakeholders and their relationships.

show abstract

Fidelity, Feasibility and Adaptation of a Family Planning Intervention for Young Women in Zimbabwe: Provider Perspectives and Experiences

Mavodza

Bernays

Mackworth-Young

et al. 2023

Glob Implement Res Appl

View full text Add to dashboard Cite

The CHIEDZA (Community-based Interventions to improve HIV outcomes in youth: a cluster randomised trial in Zimbabwe) trial evaluated an integrated package of HIV and sexual and reproductive health services for young people aged 16–24 years in Zimbabwe. The family planning component aimed to improve access to information, services, and contraceptives delivered by trained youth-friendly providers within a community-based setting for young women. Responsively adapting the intervention was a part of the intervention design’s rationale. We investigated the factors influencing implementation fidelity, quality, and feasibility using provider experiences and perspectives. We conducted provider interviews (N = 42), non-participant (N = 18), and participant observation (N = 30) of intervention activities. The data was analyzed thematically. CHIEDZA providers were receptive to providing the family planning intervention, but contexts outside of the intervention created challenges to the intervention’s fidelity. Strategic adaptations were required to ensure service quality within a youth-friendly context. These adaptations strengthened service delivery but also resulted in longer wait times, more frequent visits, and variability of Long-Acting Reversible contraceptives (LARCS) provision which depended on target-driven programming by partner organization. This study was a practical example of how tracking adaptations is vital within process evaluation methods in implementation science. Anticipating that changes will occur is a necessary pre-condition of strong evaluations and tracking adaptations ensures that lessons on feasibility of design, contextual factors, and health system factors are responded to during implementation and can improve quality. Some contextual factors are unpredictable, and implementation should be viewed as a dynamic process where responsive adaptations are necessary, and fidelity is not static.Trial registration ClinicalTrials.gov Identifier: NCT03719521.

show abstract

Timely evaluation in international development

Cited by 6 publications

References 64 publications

Challenges and opportunities in evaluating programmes incorporating human-centred design: lessons learnt from the evaluation of Adolescents 360

Challenges and opportunities in evaluating programmes incorporating human-centred design: lessons learnt from the evaluation of Adolescents 360

Engaging stakeholders with evidence and uncertainty: developing a toolkit

Fidelity, Feasibility and Adaptation of a Family Planning Intervention for Young Women in Zimbabwe: Provider Perspectives and Experiences

Contact Info

Product

Resources

About