Ten years of implementation outcomes research: a scoping review

Abstract

Background Proctor and colleagues’ 2011 paper proposed a taxonomy of eight implementation outcomes and challenged the field to address a research agenda focused on conceptualization, measurement, and theory building. Ten years later, this paper maps the field’s progress in implementation outcomes research. This scoping review describes how each implementation outcome has been studied, research designs and methods used, and the contexts and settings represented in the current literature. We also describe the role of implementation outcomes in relation to implementation strategies and other outcomes. Methods Arksey and O’Malley’s framework for conducting scoping reviews guided our methods. Using forward citation tracing, we identified all literature citing the 2011 paper. We conducted our search in the Web of Science (WOS) database and added citation alerts sent to the first author from the publisher for a 6-month period coinciding with the WOS citation search. This produced 1346 titles and abstracts. Initial abstract screening yielded 480 manuscripts, and full-text review yielded 400 manuscripts that met inclusion criteria (empirical assessment of at least one implementation outcome). Results Slightly more than half (52.1%) of included manuscripts examined acceptability. Fidelity (39.3%), feasibility (38.6%), adoption (26.5%), and appropriateness (21.8%) were also commonly examined. Penetration (16.0%), sustainability (15.8%), and cost (7.8%) were less frequently examined. Thirty-two manuscripts examined implementation outcomes not included in the original taxonomy. Most studies took place in healthcare (45.8%) or behavioral health (22.5%) organizations. Two-thirds used observational designs. We found little evidence of progress in testing the relationships between implementation strategies and implementation outcomes, leaving us ill-prepared to know how to achieve implementation success. Moreover, few studies tested the impact of implementation outcomes on other important outcome types, such as service systems and improved individual or population health. Conclusions Our review presents a comprehensive snapshot of the research questions being addressed by existing implementation outcomes literature and reveals the need for rigorous, analytic research and tests of strategies for attaining implementation outcomes in the next 10 years of outcomes research.

A community-based mentoring scheme for pregnant and parenting adolescents in Sierra Leone: protocol for a hybrid pilot cluster randomised controlled trial

“…Implementation and contextual factors (barriers and facilitators) will be further assessed using the Consolidated Framework for Implementation Research [30] and Proctor's framework [31,32]. A 13 secondary grounded theory analysis will be undertaken to generate theory to explore and explain the phenomenon of adolescent pregnancy in Sierra Leone [33].…”

What is the power of a genomic multidisciplinary team approach? A systematic review of implementation and sustainability

“…Proctor’s evaluative framework consists of healthcare service (Efficiency, safety, effectiveness, equity, patient centeredness, timeliness) and implementation (acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, sustainability) outcomes. These are well-used constructs in the implementation science literature since 2011, in over 400 papers as a standard measure of implementation outcomes [ 19 ]. These measures, and their definitions in the genomic MDT context, were further characterized by GMIR domain mapping to understand the genomic MDT outcomes (Supp table S 2 ) by two authors prior to extraction.…”

Are codesigned programmes more difficult to implement? A qualitative study of staff perceptions on the implementation of a new youth mental health programme

“…Reviews of the current mental health system in Australia, and the state of Victoria in particular, have highlighted the need to consider how mental health services are designed, implemented and delivered. 10,19,20 The 2023 Victorian Royal Commission into the Mental Health System reported that the system required significant reform. 19,20 A major recommendation of the commission was to embed codesign into mental health interventions and treatment programmes.…”

“…10,19,20 The 2023 Victorian Royal Commission into the Mental Health System reported that the system required significant reform. 19,20 A major recommendation of the commission was to embed codesign into mental health interventions and treatment programmes. The intervention referenced in this paper is an intense, psychosocial out-reach model of care, designed for young people aged 12-26 with persistent suicidal ideation or postsuicide attempt.…”

“…5,27 Team size can often diminish during the course of a project for reasons such as high staff turnover, lack of management support, feeling overburdened, and staff doubts over sustainability. 28,29 Each of these challenges can result in cost implications for a project which may not have been accounted for during the codesign phase.…”