Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach?

Abstract

When swift, accurate appraisal of evidence is required to inform policy concerning broad research questions, and budgetary constraints limit the employment of large research teams, researchers face a significant challenge which is sometimes met by reviewing existing systematic reviews. In this paper we highlight the challenges inherent in the reviews of reviews (RoR) method in order to advance understanding of this important tool for policy makers. Drawing on our experiences, we present, where possible, potential solutions to these challenges to demonstrate that RoRs, despite their limitations, can be useful for mediating policy-relevant evidence at speed.

Maximising the availability and use of high-quality evidence for policymaking: collaborative, targeted and efficient evidence reviews

“…Key features of this approach draw on aspects of other approaches to rapid evidence synthesis including the use of review level evidence (Caird et al, 2015) and stakeholder involvement (Moat et al, 2014, Rycroft-Malone et al, 2012. However, unlike many other approaches to rapid evidence synthesis (Ganann et al, 2010), our approach is highly transparent so readers can clearly see how studies were identified and where any bias could have been introduced.…”

“…This case study has some similarities and differences with other approaches to rapid evidence synthesis. Like the approach detailed by Caird et al (2015), we utilised high quality review level evidence to answer our questions. However, in cases where good quality review evidence is not available, our approach has additional flexibility by also identifying and including primary studies.…”

“…Another approach to evidence synthesis that also aims to provide results in a relatively short time-frame with limited resources is reviews of reviews (Caird et al, 2015). The approach described by Caird et al is particularly well suited when broad questions are being asked that necessitate the inclusion of different forms of evidence (e.g., reviews of trials as well as metasynthesis of views and experiences).…”

“…Like rapid realist reviews (Saul et al, 2013), our approach involved a high level of stakeholder engagement throughout the process which helped to address the barriers to getting evidence into policy. We also attempted to address some of the limitations of rapid realist reviews (see Saul et al, 2013), which have previously made compromises on rigour (i.e., not conducting a literature search) or timeliness (i.e requiring a 6 month timescale) by utilising high quality processed evidence in the forms of reviews of reviews (Caird et al, 2015;Moat et al, 2014). In addition, we used components of rapid reviews to identify primary studies (in the case of absence of systematic reviews) such as highly focused database searching and single person study selection and data extraction (Grant Booth, 2009).…”

“…In addition, relevant clinical guidelines published by the National Institute for Health and Care (NICE) and the Scottish Intercollegiate Guidelines Network (SIGN) and any guidance/ reports provided by professional organisations (e.g., the Royal College of Obstetricians and Gynaecologists, Royal College of Midwives) were also included. Unlike other approaches to rapid reviews which only include review level evidence (Caird et al, 2015), we had a system in place to identify pertinent primary literature in the case that no highly processed evidence was available. This was identified through searching reference lists of non-systematic reviews identified through the search strategy outlined above and recommendations of review sponsors.…”