The effectiveness of evidence summaries on health policymakers and health system managers use of evidence from systematic reviews: a systematic review

Abstract

BackgroundSystematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which make them hard to use for decision-making. There are many organizations that develop and disseminate derivative products, such as evidence summaries, from systematic reviews for different populations or subsets of decision makers. This systematic review aimed to (1) assess the effectiveness of evidence summaries on policymakers’ use of the evidence and (2) identify the most effective summary components for increasing policymakers’ use of the evidence. We present an overview of the available evidence on systematic review derivative products.MethodsWe included studies of policymakers at all levels as well as health system managers. We included studies examining any type of “evidence summary,” “policy brief,” or other products derived from systematic reviews that presented evidence in a summarized form. The primary outcomes were the (1) use of systematic review summaries in decision-making (e.g., self-reported use of the evidence in policymaking and decision-making) and (2) policymakers’ understanding, knowledge, and/or beliefs (e.g., changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g., format) of the summaries.ResultsOur database search combined with our gray literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text, and we included six studies (reported in seven papers) as well as protocols from two ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (four studies), and perceived usefulness or usability (three studies). Summary of findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for summary of findings tables, certain elements, such as reporting study event rates and absolute differences, were preferred as well as avoiding the use of footnotes.ConclusionsEvidence summaries are likely easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.Trial registrationThe protocol was published in the journal Systematic Reviews (2015;4:122)Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-016-0530-3) contains supplementary material, which is available to authorized users.

A Framework for Conceptualizing Evidence Needs of Health Systems

“…One found that decisionmakers described summary of findings tables with event rates and absolute differences as preferable. 49 The other found that mailed bulletins with synopses of systematic reviews that were distributed to clinicians were associated with some practice improvement. 50 Wye 2015 studied the use of evidence by local policymaking bodies in the U.K. and noted that policymakers preferred verbal over written presentation.…”

“…In general, they preferred summaries that were brief, presented in a "graded" format (key messages, executive summary) or presented verbally. 11,18,22,35 Two systematic reviews 49,50 addressed decisionmakers preferences for various derivative products based on systematic reviews. One found that decisionmakers described summary of findings tables with event rates and absolute differences as preferable.…”

From Research Evidence to “Evidence by Proxy”? Organizational Enactment of Evidence‐Based Health Care in Four High‐Income Countries

“…The growing popularity of “decision supports,” such as checklists, protocols, and assessment routines, used as “boundary objects” aiming to overcome these knowledge boundaries (Allen ) has contributed to the greater standardization of decisions and practices at the organizational level (Rousseau and Gunia ). Other approaches synthesize bodies of evidence to provide actionable advice for decision makers, often in the form of structured summaries of research (Crowley and Scott ; Newman, Cherney, and Head ; Petkovic et al ). At the same time, excessive standardization has the potential to generate professional resistance (Martin et al ) and limit responsiveness to the ever‐changing context if it is used without reflection (Kislov et al ).…”

Acesso à informação para tomada de decisão com base em evidências segundo a percepção de Secretários Municipais de Saúde do Estado do Paraná, no ano de 2014

“…Por outro lado, os pesquisadores poderiam disseminar resultados de pesquisas empregando meios adequados ao público de interesse 8 . Portanto, além dos artigos científicos, os pesquisadores poderiam empregar outras formas e meios de disseminação do conhecimento, como infográficos, reuniões com a comunidade, vídeos e seminários de treinamentos 9 .…”