RAISE: A Management and Organizational Sustainability Tool for Local Governments to Systematically Self-Evaluate the Effectiveness of Their Programs

Abstract

Context: Many donor-driven public health programs are now seeking evidence for sustainability prior to investment, creating the need for tools to better appraise these capabilities. Assessing the sustainability of programs and interventions at the local level remains a community-wide challenge. Program: This article presents a new self-assessment tool, the Reflection and Action to Improve Self-reliance and Effectiveness (RAISE) Tool ("the Tool"), modeled after The Challenge Initiative's (TCI) Sustainability Pillars. It describes the evolution of the Tool, explores its structure and applications, demonstrates its data analysis capabilities, and illustrates how it can be used for continuous program self-assessment by local governments, which TCI considers an indicator of program sustainability at the local level. Implementation: Developed in 2019, the Tool has been adapted, adopted, and implemented by 92 local governments across 11 countries. The Challenge Initiative works with these local governments over a minimum of 3 years, providing management and technical coaching on high-impact interventions. Using the Tool, local governments self-assess and evaluate the quality and effectiveness of their activity implementation and identify gaps for improvement. The Tool helps both local governments and TCI track their readiness toward becoming self-reliant and taking ownership of their family planning programs. Evaluation: As of June 30, 2021, 39 of the 92 local governments reached the final stage of maturity, self-reliance. Discussion: Experts have stated that it can take 15 years for a sustainability assessment tool such as RAISE to be adopted into government policies. After 2 years of using the Tool on a quarterly basis, on average 87.3% of eligible local governments completed the self-assessments, made course corrections, and have taken steps toward program independence. The 39 local governments that successfully progressed to self-reliance continue to use the Tool without TCI's coaching support and have expressed interest in adapting the Tool for other health interventions.

A Platform for Sustainable Scale: The Challenge Initiative’s Innovative Approach to Scaling Proven Interventions

“…TCI’s key outcome measures are anchored in platform scale, contraceptive uptake, health systems strengthening for sustainability, and platform cost-efficiency. Data is triangulated from several sources to collect these measures: the government’s service statistics as captured in its health management information system (HMIS); project records and available surveys; TCI’s RAISE (Reflection and Action to Improve Self-Reliance and Effectiveness) maturity model tool, which uses a participatory approach to self-assess sustainability progress in real time 28 ; the Most Significant Change (MSC) technique, 29 which supports adaptive management, routine monitoring, and cross-learning efforts locally and globally; and regular “pause-and-reflect” sessions that generate additional insights and evidence on the scaling pathway. In a large-scale program such as TCI, high-frequency program monitoring is critical to determine where further support is needed and where lessons can be learned, shared, and successfully replicated.…”

Improving State Government's Responsiveness to Family Planning Interventions in Nigeria Using an Innovative Reflection and Action Tool

“…It is currently being used by all 13 TCI-supported states in Nigeria as well as 97 TCI-supported cities across East Africa, Francophone West Africa, India, and the Philippines. 24 The aim is that state and city governments will continue to use this RF tool and process themselves following graduation.…”

How The Challenge Initiative Adapted and Used Pause and Reflect Responsive Feedback Sessions for Adaptive Management in Nigeria

“…Specifically, TCI has adopted a “learning by doing” approach that includes regularly scheduled P&R sessions in concert with other adaptive management approaches to monitor and provide insights into the programmatic context of program impacts across 111 cities in 11 countries. This includes the ongoing collection of Most Significant Change stories, a complexity-aware monitoring and evaluation technique that compliments performance monitoring by tracking the unpredictable 1 (described in detail in Ohkubo et al 5 ) and the implementation of the Reflection and Action to Improve Self-reliance and Effectiveness (RAISE) self-assessment tool (described in detail in Malik et al 14 and Ajijola et al 15 ). Most public health programs in Nigeria do not use RF and/or adaptive management approaches to iteratively improve programs throughout their life cycle.…”

“…12 It is critical to have tools and processes to iteratively learn, adapt, and change as needed when scaling interventions. [12][13][14] Therefore, TCI uses RF as an essential tool for adaptive management to monitor implementation, institutionalization, continuous improvement, and scale-up of government-led FP/ RH interventions. Specifically, TCI has adopted a "learning by doing" approach that includes regularly scheduled P&R sessions in concert with other adaptive management approaches to monitor and provide insights into the programmatic context of program impacts across 111 cities in 11 countries.…”