Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

Abstract

Background The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. Methods There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments. Results Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six ‘core elements’ that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout. Limitations The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development. Conclusions This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as ‘events in systems’ that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers. Future work We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources. Funding This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

What evidence is needed to demonstrate the beneficial effects of exercise for osteoarthritis?

“…Education and exercise treatment consists of several components, which interacts, depends on behaviours, expertise and skills of those delivering and those receiving it, delivered by different healthcare organisations, and is flexible and tailored to the needs of the patients. 17 Research evaluating complex interventions encounter many challenges and difficulties as acknowledged by the UK Medical Research Council. 11 18 Therefor a framework was published, which was recently updated, for the development and evaluation of complex interventions.…”

“…11 18 Therefor a framework was published, which was recently updated, for the development and evaluation of complex interventions. 17 Although it was acknowledged that no novel design exists that caters for 'complex interventions', it was emphasised that more options exist to design a study than randomised controlled trials, and guidance was given on the function of the intervention and usefulness of evidence. 17 Use of pragmatic randomised clinical trials investigating effectiveness or real world data could be helpful.…”

“…17 Although it was acknowledged that no novel design exists that caters for 'complex interventions', it was emphasised that more options exist to design a study than randomised controlled trials, and guidance was given on the function of the intervention and usefulness of evidence. 17 Use of pragmatic randomised clinical trials investigating effectiveness or real world data could be helpful. Six core elements have been described that are crucial in development and evaluation: context, programme theory, stakeholders, uncertainty, intervention refinement and economic considerations.…”

“We threw away the stones”: a mixed method evaluation of a simple cookstove intervention in Malawi

“…The combination of qualitative and quantitative enquiry, with each applied as appropriate, was used here as it allows for a fuller exploration of outcomes, particularly important for complex interventions with social elements 34,35 . Rather than separate but parallel applications and analysis, an integrated synthesis was used, allowing for more in-depth findings than when either single methodology is used alone.…”

Framework for identification and measurement of spillover effects in policy implementation: intended non-intended targeted non-targeted spillovers (INTENTS)

“…Among researchers and policymakers, there is increasing awareness of the potential for the implementation of health interventions and policies to have consequences beyond those initially intended. For example, the Medical Research Council guidance on complex interventions emphasises that these unintended consequences should be considered where possible but provides little guidance on how best to do this systematically [ 1 ]. This lack of direction is also evident in the implementation science literature.…”