Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews

Abstract

BackgroundLogic models are becoming an increasingly common feature of systematic reviews, as is the use of programme theory more generally in systematic reviewing. Logic models offer a framework to help reviewers to ‘think’ conceptually at various points during the review, and can be a useful tool in defining study inclusion and exclusion criteria, guiding the search strategy, identifying relevant outcomes, identifying mediating and moderating factors, and communicating review findings.Methods and FindingsIn this paper we critique the use of logic models in systematic reviews and protocols drawn from two databases representing reviews of health interventions and international development interventions. Programme theory featured only in a minority of the reviews and protocols included. Despite drawing from different disciplinary traditions, reviews and protocols from both sources shared several limitations in their use of logic models and theories of change, and these were used almost unanimously to solely depict pictorially the way in which the intervention worked. Logic models and theories of change were consequently rarely used to communicate the findings of the review.ConclusionsLogic models have the potential to be an aid integral throughout the systematic reviewing process. The absence of good practice around their use and development may be one reason for the apparent limited utility of logic models in many existing systematic reviews. These concerns are addressed in the second half of this paper, where we offer a set of principles in the use of logic models and an example of how we constructed a logic model for a review of school-based asthma interventions.

What have we learned after ten years of systematic reviews in international development?

“…There have been a number of calls for the incorporation of programme theory into systematic reviews over the years (for example, Pawson 2002;Davies 2006;van der Knaap et al, 2008;Anderson et al 2011;Waddington et al 2012;Snilstveit 2012;Kneale, Thomas, and Harris 2015;Maden et al 2017;White 2018), as well as for multi-disciplinary working (for example, Thomas et al 2004;Snilstveit 2012;Oliver et al 2017;White 2018). Programme theory is usually incorporated into systematic reviews through logic models (flow diagrams which present the intervention causal chain from inputs through to final outcomes) or theories of change (which articulate the assumptions underlying the causal chain and the contexts and stakeholders for whom the intervention is relevant), and sometimes through economic, social or psychological theory to help articulate programme mechanisms.…”

Logic Model Evaluation For SMK / SMA Teachers in Dual Expertise Educational Certification Program

“…The Logic Model helps explain how and why the program works; [2]. Logic Model can be used as a tool to improve planning, implementation and dissemination of Dual Expertise Education Certification Program for SMK / SMA Teachers; and [3]. Logic Models can help connect the need for a planned program to the desired outcome of the program.…”

“…Developing a logic model evaluation is an important first step in program evaluation [9]. As a pragmatic tool carrying out the review, strengthening the quality of reviews, communication tools dissemination findings [3]. Aim to provide street map stakeholders describing the sequence of related events that link the needs of the planned program to the desired outcomes of the program.…”

Cochrane Qualitative and Implementation Methods Group guidance series—paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings

“…Logic models can play a role in articulating programme theory by "representing the underlying processes by which an intervention effects a change on individuals, communities and organisations" [15]. Logic models can be expressed visually, usually in the form of a diagram showing the links between intervention inputs and outputs.…”

“…Process evaluations are designed to assess a variety of questions including assessing intervention fidelity and reach, barriers to implementation and participant and provider experiences of the intervention [14]. The findings of qualitative research and process evaluations can be used to build programme theory [14][15]. A common approach conceptualises programme theory as comprised of a theory of action and a theory of change, both of which are shaped by broader M A N U S C R I P T…”