Synthesis for health services and policy: case studies in the scoping of reviews

Abstract

Background For systematic reviews to be rigorous, deliverable and useful, they need a well-defined review question. Scoping for a review also requires the specification of clear inclusion criteria and planned synthesis methods. Guidance is lacking on how to develop these, especially in the context of undertaking rapid and responsive systematic reviews to inform health services and health policy. Objective This report describes and discusses the experiences of review scoping of three commissioned research centres that conducted evidence syntheses to inform health and social care organisation, delivery and policy in the UK, between 2017 and 2020. Data sources Sources included researcher recollection, project meeting minutes, e-mail correspondence with stakeholders and scoping searches, from allocation of a review topic through to review protocol agreement. Methods We produced eight descriptive case studies of selected reviews from the three teams. From case studies, we identified key issues that shape the processes of scoping and question formulation for evidence synthesis. The issues were then discussed and lessons drawn. Findings Across the eight diverse case studies, we identified 14 recurrent issues that were important in shaping the scoping processes and formulating a review’s questions. There were ‘consultative issues’ that related to securing input from review commissioners, policy customers, experts, patients and other stakeholders. These included managing and deciding priorities, reconciling different priorities/perspectives, achieving buy-in and engagement, educating the end-user about synthesis processes and products, and managing stakeholder expectations. There were ‘interface issues’ that related to the interaction between the review team and potential review users. These included identifying the niche/gap and optimising value, assuring and balancing rigour/reliability/relevance, and assuring the transferability/applicability of study evidence to specific policy/service user contexts. There were also ‘technical issues’ that were associated with the methods and conduct of the review. These were choosing the method(s) of synthesis, balancing fixed and fluid review questions/components/definitions, taking stock of what research already exists, mapping versus scoping versus reviewing, scoping/relevance as a continuous process and not just an initial stage, and calibrating general compared with specific and broad compared with deep coverage of topics. Limitations As a retrospective joint reflection by review teams on their experiences of scoping processes, this report is not based on prospectively collected research data. In addition, our evaluations were not externally validated by, for example, policy and service evidence users or patients and the public. Conclusions We have summarised our reflections on scoping from this programme of reviews as 14 common issues and 28 practical ‘lessons learned’. Effective scoping of rapid, responsive reviews extends beyond information exchange and technical procedures for specifying a ‘gap’ in the evidence. These considerations work alongside social processes, in particular the building of relationships and shared understanding between reviewers, research commissioners and potential review users that may be reflective of consultancy, negotiation and co-production models of research and information use. Funding This report has been based on work commissioned by the National Institute for Health Research (NIHR) Health Services and Delivery Research (HSDR) programme as three university-based evidence synthesis centres to inform the organisation, delivery and commissioning of health and social care; at the University of Exeter (NIHR 16/47/22), the University of Sheffield (NIHR 16/47/17) and the University of York (NIHR 16/47/11). This report was commissioned by the NIHR HSDR programme as a review project (NIHR132708) within the NIHR HSDR programme. This project was funded by the NIHR HSDR programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 15. See the NIHR Journals Library website for further project information.

Rapid evaluation of service innovations in health and social care: key considerations

“…Useful insights about and examples of evidence synthesis in health and social care research have been set out in a report from the NIHR Evidence Synthesis Centres. 94 Evidence reviews can be integrated into different stages of rapid evaluations, often being used to inform a scoping exercise by mapping the available published research evidence in relation to the wider context or prior implementation of a similar intervention. These reviews can also help researchers understand how these interventions have been evaluated in the past and use these insights to inform the rapid evaluation scope and design.…”

A systematic review case study of urgent and emergency care configuration found citation searching of Web of Science and Google Scholar of similar value

“…It is one of three review facilities, commissioned on a 3-year renewable contract to conduct literature reviews within time-sensitive windows, to inform both policy and future research (Chambers, Booth, et al, 2021). Future topics are discussed at an annual performance review meeting with members of the NIHR HS&DR Programme team or, subsequently, surface as urgent priorities throughout the rest of the year (Anderson et al, 2021). Review products range from desk based surveys of ongoing research, concluded within three or four working days, to full-scale systematic reviews where multiple reviewers are employed to accelerate the review process.…”