A longitudinal mixed method approach for assessing implementation context and process factors: Comparison of three sites from a Housing First implementation strategy pilot

Abstract

Background: Implementation science’s focus on establishing implementation strategy effectiveness has overshadowed the need to understand differential performance of such strategies under various conditions. Methods allowing for assessment between implementation context and process can help address this gap. This article provides a detailed description of a mixed method procedure for assessing factors related to the implementation context and process intersection, which was developed as part of the pilot study of the Housing First Technical Assistance and Training (HFTAT) program, a multifaceted strategy designed to support Housing First model implementation. Methods: The HFTAT was pilot tested among a sample of three organizations. Our mixed method approach combines two tools often used in implementation research—the Stages of Implementation Completion (SIC) and the Consolidated Framework for Implementation Research—in a novel way. Several stages to analysis were completed, starting with a separate analysis of data pertaining to each measure and then two levels of mixed method analysis. Results: The approach provided a better understanding of the issues that impacted the implementation guided by the HFTAT, suggesting (1) individual determinants seemed to have a bigger impact based on the number of SIC phases they affected, (2) implementation context and process were connected through climate-related factors in the inner setting that made the sites more or less responsive to addressing identified barriers, and (3) there is a need to better assess context factors to identify areas where implementation drivers should be better targeted to facilitate change, and this is supported by prior research. Conclusions: Understanding the underlying factors impacting a setting’s performance related to a specific implementation strategy has potential to improve decision-making and optimize future implementation efforts. The approach likely be as successful combining the SIC with other determinant frameworks and should be utilized at the onset of an implementation project to maximize its usefulness. Plain language summary The field of implementation science needs better approaches for understanding how context (i.e., constraints and opportunities in the implementation setting) and process (i.e., the actions and steps taken during implementation) interact over the course of implementation. Such information would be very useful for understanding the success or failure of specific implementation strategies, which are specific techniques used to increase the adoption, implementation, and sustainability of a specific intervention. This article explains a method to assess the intersection of implementation context and implementation process that we developed to better understand issues that could help explain the effectiveness of an implementation strategy for an intervention for housing people who are experiencing chronic homelessness and who have both a diagnosed substance use disorder and serious mental illness. The method combines two popular implementation tools, the Stages of Implementation Completion and the Consolidated Framework for Implementation Research. Using this method resulted in a better understanding of differences in implementation performance at each of the organizations and how we might improve the implementation strategy. This understanding was better than what we had learned from other approaches we had used before this. We provide some suggestions for how the method can be strengthened for use in other studies.

The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs

“…Each of the three Pre-Implementation stages are positively, significantly, and independently associated with program start-up. Although the Pre-Implementation phase (across all of the first three stages) typically stands alone (e.g., [ 16 , 21 ]) and was shown in the current study to be predictive of program start-up, the relative importance of each stage was remarkable. This provides further evidence that each of the three stages measures different aspects of Pre-Implementation fidelity [ 25 ].…”

Stepped implementation-to-target: a study protocol of an adaptive trial to expand access to addiction medications

“…A third Final Stage score indicates the final stage achieved in the implementation process (Stages 1-8). The SIC has been adapted or customized for multiple evidence-based practices (EBPs) [70][71][72][73][74][75] and evaluated across different system settings [51,76,77].…”

Barriers impacting the POINT pragmatic trial: the unavoidable overlap between research and intervention procedures in “real-world” research

“…We employed the Consolidated Framework for Implementation Research (CFIR) as the guiding framework for our qualitative inquiry of challenges impacting the POINT trial. We selected the CFIR as the guiding implementation framework because it was developed to be applied specifically within a clinical context and offers a highly detailed list of domains and constructs that lend themselves well to deductive qualitative work [16][17][18]. The CFIR comprises 37 implementation determinant constructs organized in 5 domains: inner setting, outer setting, intervention characteristics, characteristics of individuals, and the implementation process.…”