Cost-effectiveness of the Adaptive Implementation of Effective Programs Trial (Adept): Approaches to Adopting Implementation Strategies

Abstract

Background Theory-based methods to support clinician uptake of evidence-based practices (EBPs) are critical to improving mental health outcomes. Costs associated with effective implementation strategies can be substantial, and few have been rigorously evaluated. The purpose of this study is to conduct a cost-effectiveness analysis to identify the most cost-effective approach to deploying implementation strategies to enhance the uptake of Life Goals, a mental health EBP. Methods We used data from a previously conducted randomized trial to compare the cost-effectiveness of different augmentations to Replicating Effective Programs (REP) combined with external and/or internal facilitation to enhance uptake of Life Goals. REP is a low-level strategy that includes EBP packaging, training, and technical assistance. External facilitation (EF) involves external expert support, and internal facilitation (IF) augments EF with protected time for internal staff to support EBP implementation. We developed a decision tree to assess 1-year costs and outcomes for four implementation strategies: 1) REP only, 2) REP + EF 3) REP + EF add IF if necessary, 4) REP + EF/IF. The analyses used a 1-year time horizon and assumes a health payer perspective. Our outcome was quality-adjusted life years (QALYs). The economic outcome was the incremental cost-effectiveness ratio (ICER). We conducted deterministic and probabilistic sensitivity analysis (PSA). Results Our results indicate that REP + EF add IF is the most cost-effective option with an ICER of $593/QALY. The REP + EF/IF and REP + EF only conditions are dominated. One-way sensitivity analyses indicate that results are sensitive to utilities for REP + EF and REP + EF add IF. The PSA results indicate that REP + EF, add IF is the optimal strategy in 30% of iterations at the threshold of $100,000/QALY. Conclusions Our results suggest that the most cost-effective implementation support begins with a less intensive, less costly strategy initially and increases as needed to enhance EBP uptake. Using this approach, implementation support resources can be judiciously allocated to those clinics that would most benefit. Our results were not robust to changes in the utility measure. Further research is needed that incorporates robust and relevant utilities in implementation research to identify the most cost-effective strategies. This research advances economic evaluation of implementation by assessing costs and utilities across multiple implementation strategy combinations. Trial registration: ClinicalTrials.gov Identifier: NCT02151331, 05/30/2014

The economics of adaptations to evidence-based practices

“…Guided by the FRAME and other tools [20], mixed methods approaches can be highly useful to elucidate the economic reasons driving the adaptations to improve the generalizability of findings, as well as their economic consequences [21]. Mixed methods can also inform sensitivity analysis that is often a critical component to test the assumptions made in economic evaluation [22], by further identifying and explaining variations in cost at different levels (e.g., provider, practice) and for various stakeholders. For implementation planning purposes, it is useful to understand the dimensions upon which planned adaptations may be warranted and to establish a range in costs associated with planned adaptations that are likely to occur.…”

“…The trial evaluated implementation strategies of varying intensity, with a lowintensity strategy that includes EBP packaging, training, and technical assistance, followed by a medium-intensity strategy involving facilitation by external experts, and a high-intensity strategy involving internal facilitation with protected time for internal staff to support EBP implementation. Implementation strategies were evaluated for their cost-effectiveness through random assignment in three stages, whereby sites not responding to the implementation strategy in the previous stage were further randomized to receive a supplemental implementation strategy in the subsequent stage [22]. Findings suggested that the most cost-effective implementation support starts with a less intensive, less costly implementation strategy and increases as needed to enhance EBP uptake [22].…”

“…Implementation strategies were evaluated for their cost-effectiveness through random assignment in three stages, whereby sites not responding to the implementation strategy in the previous stage were further randomized to receive a supplemental implementation strategy in the subsequent stage [22]. Findings suggested that the most cost-effective implementation support starts with a less intensive, less costly implementation strategy and increases as needed to enhance EBP uptake [22]. A SMART design is also being applied to the implementation of clinical guidelines for opioid prescribing in primary care settings, in which clinics are randomized to receive a sequence of implementation strategies that address implementation concerns at the health system, clinic, and provider levels [32].…”

Applying concepts from “rapid” and “agile” implementation to advance implementation research

“…The study design aimed to determine, among clinics that had been non-responsive to REP, the cost-effectiveness of adding external or internal facilitation as supplemental implementation strategies. A cost-effectiveness analysis found that the adaptive strategy that begins with a less intensive, less costly strategy, and increases implementation support based on monitoring the implementation response variable, was the most cost-effective strategy among those tested in the trial [ 45 ]. Resources such as those available at the NIH-funded Methodology Center at Pennsylvania State University [ 46 ] can help implementation researchers design studies that are both statistically rigorous and adaptive in their approach.…”

Michigan Model for HealthTM Learning to Enhance and Adapt for Prevention (Mi-LEAP): protocol of a pilot randomized trial comparing Enhanced Replicating Effective Programs versus standard implementation to deliver an evidence-based drug use prevention curriculum

“…One approach is a modified cost calculator approach that has been applied in the Costs of Implementing New Strategies (COINS); this approach identifies a range of costs across phases of implementation (e.g., pre-implementation, implementation, and sustainability) tailored to the strategies utilized for a specific implementation effort and focused on the perspective of the organization/provider deciding to adopt the EBI [63][64][65][66]. This is useful in identifying costs related to implementation for several reasons: (1) it aids in determining direct costs of implementation through tallying time spent on activities in each phase of implementation strategy deployment (often the bulk of implementation strategy costs), (2) this practical approach can provide needed guidance and scaffolding for stakeholders and decision-makers to determine implementation costs so organizations could accurately estimate the necessary resources for implementation success, and (3) this approach has been used previously with Enhanced REP in estimating costs as the first step in cost-effectiveness analysis for a community-based clinical trial [67]. Table 4 provides a list of anticipated activities whose costs will be estimated prospectively as part of the pilot trial.…”