Synthesising quantitative evidence in systematic reviews of complex health interventions

Abstract

Public health and health service interventions are typically complex: they are multifaceted, with impacts at multiple levels and on multiple stakeholders. Systematic reviews evaluating the effects of complex health interventions can be challenging to conduct. This paper is part of a special series of papers considering these challenges particularly in the context of WHO guideline development. We outline established and innovative methods for synthesising quantitative evidence within a systematic review of a complex intervention, including considerations of the complexity of the system into which the intervention is introduced. We describe methods in three broad areas: non-quantitative approaches, including tabulation, narrative and graphical approaches; standard meta-analysis methods, including meta-regression to investigate study-level moderators of effect; and advanced synthesis methods, in which models allow exploration of intervention components, investigation of both moderators and mediators, examination of mechanisms, and exploration of complexities of the system. We offer guidance on the choice of approach that might be taken by people collating evidence in support of guideline development, and emphasise that the appropriate methods will depend on the purpose of the synthesis, the similarity of the studies included in the review, the level of detail available from the studies, the nature of the results reported in the studies, the expertise of the synthesis team and the resources available.

Prevention and control of cholera with household and community water, sanitation and hygiene (WASH) interventions: A scoping review of current international guidelines

“…None of the guidelines explicitly stated their process for guideline development such as using the GRADE system [151,152] or other recommended methods [93,150,153,154] to determine the quality of evidence for each recommendation. Any new development of guidelines should either use and adhere to these recommended processes to strengthen their quality and use, or clearly describe their methods.…”

Proposed pathway for patients undergoing enhanced recovery after spinal surgery: protocol for a systematic review

“…It will need to be applied in a dynamic multidimensional environment. The planned review will therefore be performed according to the methodological standards for complex reviews [21][22][23][24][25][26][27][28].…”

“…Summary of findings table will not be constructed [40]. Narrative analysis of each pathway component will present the summarized data used to inform the synthesis in the form of a forest plot [27]. Forest plot will be used in order to summarize data for each relevant outcome of each pathway component.…”

“…Meta-analyses will not be conducted due to the anticipated inherent methodological heterogeneity of the studies with regard to design, populations, procedures, and outcomes [43]. As such, it is unlikely that quantitative synthesis would be meaningful in this review of complex intervention [27].…”

“…Quality of evidence can be seen as occurring in a continuum; therefore, any discrete categorization of continuous evidence base does involve a degree of arbitrariness. As we are undertaking a systematic review, rather than developing a guideline, we will not be making recommendations on the utility of the pathway components [27,40].…”