Selecting and implementing overview methods: implications from five exemplar overviews

Abstract

BackgroundOverviews of systematic reviews are an increasingly popular method of evidence synthesis; there is a lack of clear guidance for completing overviews and a number of methodological challenges. At the UK Cochrane Symposium 2016, methodological challenges of five overviews were explored. Using data from these five overviews, practical implications to support methodological decision making of authors writing protocols for future overviews are proposed.MethodsMethods, and their justification, from the five exemplar overviews were tabulated and compared with areas of debate identified within current literature. Key methodological challenges and implications for development of overview protocols were generated and synthesised into a list, discussed and refined until there was consensus.ResultsMethodological features of three Cochrane overviews, one overview of diagnostic test accuracy and one mixed methods overview have been summarised. Methods of selection of reviews and data extraction were similar. Either the AMSTAR or ROBIS tool was used to assess quality of included reviews. The GRADE approach was most commonly used to assess quality of evidence within the reviews.Eight key methodological challenges were identified from the exemplar overviews. There was good agreement between our findings and emerging areas of debate within a recent published synthesis. Implications for development of protocols for future overviews were identified.ConclusionsOverviews are a relatively new methodological innovation, and there are currently substantial variations in the methodological approaches used within different overviews. There are considerable methodological challenges for which optimal solutions are not necessarily yet known. Lessons learnt from five exemplar overviews highlight a number of methodological decisions which may be beneficial to consider during the development of an overview protocol.Electronic supplementary materialThe online version of this article (doi:10.1186/s13643-017-0534-3) contains supplementary material, which is available to authorized users.

An introduction to overviews of reviews: planning a relevant research question and objective for an overview

“…Following best practice, these aspects should be covered within a published overview protocol as a mechanism for ensuring transparency and reducing opportunities for introduction of bias in the conduct of the overview. Our second paper [5] outlines a number of key methodological decisions which we consider important to address when planning an overview, and which will be important to incorporate within an overview protocol.…”

“…Overviews are a relatively new and emerging method of summarising evidence, and consequently universally-accepted guidance for good practice relating to the conduct of overviews is currently lacking [5, 13–17]. During a 2016 UK Cochrane Symposium workshop [18] focused on the methods and challenges associated with overviews, it became apparent that there was a need to clarify, and distinguish between, different types of overviews and the objectives which these overviews addressed.…”

“…Within this paper, we therefore describe types of overview and the common research questions and objectives they address. Within a second, linked paper [5], we build on this description of overview types, objectives and research questions, illustrating this through the use of five exemplar overviews, and exploring the impact and implications of different methodological approaches.…”

“…In presenting and discussing common research questions addressed by overviews with different objectives, and relating this to real examples in the second paper [5], we aim to help readers understand important issues to consider during the first steps to planning an overview.…”

“…There are a range of factors to reflect upon prior to deciding whether to conduct an overview, including consideration of the methodological challenges and uncertainties. These challenges are discussed in depth in our accompanying paper on this topic [5]. …”