A framework for learning about improvement: embedded implementation and evaluation design to optimize learning

Abstract

Improving health care involves many actors, often working in complex adaptive systems. Interventions tend to be multi-factorial, implementation activities diverse, and contexts dynamic and complicated. This makes improvement initiatives challenging to describe and evaluate as matching evaluation and program designs can be difficult, requiring collaboration, trust and transparency. Collaboration is required to address important epidemiological principles of bias and confounding. If this does not take place, results may lack credibility because the association between interventions implemented and outcomes achieved is obscure and attribution uncertain. Moreover, lack of clarity about what was implemented, how it was implemented, and the context in which it was implemented often lead to disappointment or outright failure of spread and scale-up efforts. The input of skilled evaluators into the design and conduct of improvement initiatives can be helpful in mitigating these potential problems. While evaluation must be rigorous, if it is too rigid necessary adaptation and learning may be compromised. This article provides a framework and guidance on how improvers and evaluators can work together to design, implement and learn about improvement interventions more effectively.

The time has come: Embedded implementation research for health care improvement

“…This might range from organically developed partnerships, through to a more formal, funded investment in a researcher co‐located and working within a health service . Either will involve a strong measure of collaboration between implementers (in various roles such as policymakers, decision makers, or frontline staff) and researchers (in various roles such as implementation scientists, methodologists, or evaluators) from the start of an implementation effort, sometimes by co‐locating the researcher with the implementers …”

“…The primary advantage of embedded implementation research is that efforts to introduce a change now sit at the intersection of implementers' appreciation of change and capacity to influence context on the one hand, and researchers' knowledge and expertise in empirical methods on the other. An embedded approach differs from the more traditional research on implementation that envisages disparate roles, typically uses a fixed improvement protocol, has an external researcher who provides knowledge of evidence and its use, and conducts summative evaluation . In Table , we summarize key benefits accruing to both researchers and implementers from this inherently collaborative approach.…”

“…In the programs, researchers from diverse academic backgrounds (eg, social science, operational research) are often co‐located with frontline staff to improve services and foster engagement among stakeholders. Barry et al described a global summit which sought to categorize embedded research, developing a framework to emphasize its iterative and communicative‐nature. In a recent empirical example of embedded implementation research, Alcalde‐Rabanal, Becerril‐Montekio, Langlois evaluated communities of practice (CoPs) in maternal health decision‐making in Mexico and Nicaragua.…”

“…The study arose from an initiative by a World Health Organization (WHO) partnership; indeed, embedded implementation research is becoming increasingly central to the WHO's improvement efforts in low‐ and middle‐income countries (LMIC) . Hence, as Barry et al also suggests, there is increasing consensus that embedded research holds promise in addressing some of the intractable difficulties of health system improvement experienced internationally.…”

“…Indeed, there is no clear‐cut separation between embedded or external implementation‐evaluation; rather, this distinction rests on degree of embeddedness. Here, we have devised a model, partly based on Barry et al in which four approaches to implementation or change research are depicted by extent of embeddedness (see Figure ).…”