Qualitative approaches to use of the RE-AIM framework: rationale and methods

Holtrop, Jodi Summers; Rabin, Borsika; Glasgow, Russell E.

doi:10.1186/s12913-018-2938-8

Cited by 236 publications

(247 citation statements)

References 43 publications

Supporting

Mentioning

244

Contrasting

Unclassified

Order By: Relevance

“…However, they lack clear definitions, guidance or measures to assist planning teams in understanding or improving results (15). Use of qualitative methods, such as asking stakeholders and observing processes to identify barriers to implementation, have been recommended to further our understanding of why implementers got the results they did (12,16). While anticipating barriers is important, understanding individual, situational and structural influences on outcome expectations, behavior and decision-making can identify specific mechanisms that could be assessed and addressed during implementation planning (4,15).…”

Section: Introductionmentioning

confidence: 99%

Planning for Implementation Success Using RE-AIM and CFIR Frameworks: A Qualitative Study

King

Shoup

Raebel

et al. 2020

Front. Public Health

135

105

View full text Add to dashboard Cite

Background: RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) and CFIR (Consolidated Framework for Implementation Research) dissemination and implementation frameworks define theory-based domains associated with the adoption, implementation and maintenance of evidence-based interventions. Used together, the two frameworks identify metrics for evaluating implementation success, i.e., high reach and effectiveness resulting in sustained practice change (RE-AIM), and modifiable factors that explain and enhance implementation outcomes (CFIR). We applied both frameworks to study the implementation planning process for a technology-delivered asthma care intervention called Breathewell within an integrated care organization. The goal of the Breathewell intervention is to increase the efficiency of delivering resource-intensive asthma care services. Methods: We reviewed historical documents (i.e., meeting agendas; minutes) from 14 months of planning to evaluate alignment of implementation team priorities with RE-AIM domains. Key content was extracted and analyzed on topics, frequency and amount of discussion within each RE-AIM domain. Implementation team members were interviewed using questions adapted from the CFIR Interview Guide Tool to focus their reflection on the process and contextual factors considered during pre-implementation planning. Documents and transcripts were initially coded using RE-AIM domain definitions, and recoded using CFIR constructs, with intent to help explain how team decisions and actions can contribute to adoption, implementation and maintenance outcomes. Results: Qualitative analysis of team documents and interviews demonstrated strong alignment with the RE-AIM domains: Reach, Effectiveness, and Implementation; and with the CFIR constructs: formal inclusion of provider and staff stakeholders in implementation planning, compatibility of the intervention with workflows and systems, and alignment of the intervention with organizational culture. Focus on these factors likely contributed to RE-AIM outcomes of high implementation fidelity. However, team members expressed low confidence that Breathewell would be adopted and maintained post-trial. A potential explanation was weak alignment with several CFIR constructs, including tension for change, relative priority, and leadership engagement that contribute to organizational receptivity and motivation to sustain change. King et al. Planning Using RE-AIM and CFIR Conclusions: While RE-AIM provides a practical framework for planning and evaluating practice change interventions to assure their external validity, CFIR explains why implementation succeeded or failed, and when used proactively, identifies relevant modifiable factors that can promote or undermine adoption, implementation, and maintenance.

show abstract

Section: Introductionmentioning

confidence: 99%

Planning for Implementation Success Using RE-AIM and CFIR Frameworks: A Qualitative Study

King

Shoup

Raebel

et al. 2020

Front. Public Health

135

105

View full text Add to dashboard Cite

show abstract

“…In our setting, implementation and sustainability axes heavily relied on qualitative data—the mapping out of partnerships, specialist referral corridors for patients, and other key players in the health system to engage—all these rely on narrative inquiry and interview data with stakeholders. Indeed, others have recently highlighted the use of RE‐AIM in qualitative evaluation . Overall, the comprehensiveness of an evaluation is strengthened by employing mixed‐methods approaches…”

Section: Discussionmentioning

confidence: 99%

“…Indeed, others have recently highlighted the use of RE-AIM in qualitative evaluation. 57 Overall, the comprehensiveness of an evaluation is strengthened by employing mixed-methods approaches. 58…”

Section: Measurement Toolsmentioning

confidence: 99%

Designing a program evaluation for a medical‐dental service for adults with autism and intellectual disabilities using the RE‐AIM framework

Lai

Klag

Shikako‐Thomas

2019

Learning Health Systems

View full text Add to dashboard Cite

Introduction Robust evaluation of service models can improve the quality and efficiency of care while articulating the models for potential replication. Even though it is an essential part of learning health systems, evaluations that benchmark and sustain models serving adults with developmental disabilities are lacking, impeding pilot programs from becoming official care pathways. Here, we describe the development of a program evaluation for a specialized medical‐dental community clinic serving adults with autism and intellectual disabilities in Montreal, Canada. Method Using a Participatory Action‐oriented approach, researchers and staff co‐designed an evaluation for a primary care service for this population. We performed an evaluability assessment to identify the processes and outcomes that were feasible to capture and elicited perspectives at both clinical and health system levels. The RE‐AIM framework was used to categorize and select tools to capture data elements that would inform practice at the clinic. Results We detail the process of conceptualizing the evaluation framework and operationalizing the domains using a mixed‐methods approach. Our experience demonstrated (1) the utility of a comprehensive framework that captures contextual factors in addition to clinical outcomes, (2) the need for validated measures that are not cumbersome for everyday practice, (3) the importance of understanding the functional needs of the organization and building a sustainable data infrastructure that addresses those needs, and (4) the need to commit to an evolving, “living” evaluation in a dynamic health system. Conclusions Evaluation employing rigorous patient‐centered and systems‐relevant metrics can help organizations effectively implement and continuously improve service models. Using an established framework and a collaborative approach provides an important blueprint for a program evaluation in a learning health system. This work provides insight into the process of integrating care for vulnerable populations with chronic conditions in health care systems and integrated knowledge generation processes between research and health systems.

show abstract

“…RE-AIM was selected as a guiding framework as it was developed to translate behavioral interventions into sustainable practice [49]. Full application of the model requires in-depth understanding of contextual and multilevel factors influencing implementation, that use of both qualitative and quantitative methods provide, to understand and to explain reasons for outcomes on the RE-AIM dimensions [50]. Phases were interactive, with each phase building upon prior findings, and with iterative quantitative and qualitative data collection and analyses over the three phases [48].…”

Section: Methodsmentioning

confidence: 99%

Using Mixed Methods to Adapt and Evaluate the Implementation of a Multicomponent Tobacco-Free Workplace Program within Behavioral Health Centers

Leal

Chen

Correa-Fernández

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Behavioral health treatment centers (BHTCs) rarely implement existing evidence-based practices for treating tobacco dependence, despite high rates of tobacco use among their clients. Taking Texas Tobacco Free (TTTF) has successfully targeted this disparity by delivering an evidence-based, multilevel, tobacco-free workplace program providing policy implementation and enforcement, education, provider training in tobacco screenings and treatments, and nicotine replacement therapies (NRT) to BHTCs across Texas. We describe a mixed methods design used to conduct a formative evaluation process to adapt implementation strategies to local contexts, evaluate program outcomes and characterize processes influencing program implementation in two BHTCs serving 17 clinics.Methods: Varied data collection included pre and post-implementation leader, provider, and staff surveys; and pre, mid, and post-implementation provider, staff and consumer focus groups. During implementation, data were collected via various logs (tobacco screenings, NRT delivery) to monitor program content delivery. The RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework guided translation of behavioral interventions into sustainable practice.Results: While program implementation varied between each BHTC, all clinics adopted a 100% tobacco-free workplace policy, integrated tobacco screenings into routine practice, delivered evidence-based interventions, dispensed NRT to consumers and staff, and increased provider knowledge on how to address tobacco dependence. Pre, mid, and post-implementation qualitative findings served to: 1) develop program strategies and materials adapted to local contexts and populations and address barriers; 2) adjust delivery systems of key components to enhance implementation; 3) understand reasons for success or failure to implement specific practices; and 4) reveal program integration into clinic culture, enhancing sustainability.Conclusions: Implementation of TTTF at both BHTCs increased organizational capacity in the provision of evidence-based practices to treat tobacco dependence through successfully meeting most of our RE-AIM targets. Mixing methods involved program adopters and recipients as collaborators directly shaping core interventions to their individual context and needs, thus increasing program fit, ownership, adoption and sustainability; closing the gap between research and practice. These findings contribute to the development of flexible strategies and interventions capable of addressing variable implementation contexts and barriers, thus enhancing the effectiveness and sustainability of a tobacco-free workplace program.

show abstract

Qualitative approaches to use of the RE-AIM framework: rationale and methods

Cited by 236 publications

References 43 publications

Planning for Implementation Success Using RE-AIM and CFIR Frameworks: A Qualitative Study

Planning for Implementation Success Using RE-AIM and CFIR Frameworks: A Qualitative Study

Designing a program evaluation for a medical‐dental service for adults with autism and intellectual disabilities using the RE‐AIM framework

Using Mixed Methods to Adapt and Evaluate the Implementation of a Multicomponent Tobacco-Free Workplace Program within Behavioral Health Centers

Contact Info

Product

Resources

About