Evidence based policy making and the ‘art’ of commissioning – how English healthcare commissioners access and use information and academic research in ‘real life’ decision-making: an empirical qualitative study

Abstract

BackgroundPolicymakers such as English healthcare commissioners are encouraged to adopt ‘evidence-based policy-making’, with ‘evidence’ defined by researchers as academic research. To learn how academic research can influence policy, researchers need to know more about commissioning, commissioners’ information seeking behaviour and the role of research in their decisions.MethodsIn case studies of four commissioning organisations, we interviewed 52 people including clinical and managerial commissioners, observed 14 commissioning meetings and collected documentation e.g. meeting minutes and reports. Using constant comparison, data were coded, summarised and analysed to facilitate cross case comparison.ResultsThe ‘art of commissioning’ entails juggling competing agendas, priorities, power relationships, demands and personal inclinations to build a persuasive, compelling case. Policymakers sought information to identify options, navigate ways through, justify decisions and convince others to approve and/or follow the suggested course. ‘Evidence-based policy-making’ usually meant pragmatic selection of ‘evidence’ such as best practice guidance, clinicians’ and users’ views of services and innovations from elsewhere. Inconclusive or negative research was unhelpful in developing policymaking plans and did not inform disinvestment decisions. Information was exchanged through conversations and stories, which were fast, flexible and suited the rapidly changing world of policymaking. Local data often trumped national or research-based evidence. Local evaluations were more useful than academic research.DiscussionCommissioners are highly pragmatic and will only use information that helps them create a compelling case for action.Therefore, researchers need to start producing more useful information.ConclusionsTo influence policymakers’ decisions, researchers need to 1) learn more about local policymakers’ priorities 2) develop relationships of mutual benefit 3) use verbal instead of writtencommunication 4) work with intermediaries such as public health consultants and 5) co-produce local evaluations.Electronic supplementary materialThe online version of this article (doi:10.1186/s12913-015-1091-x) contains supplementary material, which is available to authorized users.

A Framework for Conceptualizing Evidence Needs of Health Systems

“…36 Others also mentioned the need for local contextual information or applicability information, and descriptions of the uncertainty of the findings. 11,35 Such contextual information is critical for decisionmaking and is consistent with the GRADE Evidence to Decision framework, which describes several discrete factors, other than effectiveness evidence, needed to make decisions (e.g., cost effectiveness and resource use, equity, acceptability, feasibility and values). 37 In contrast, other health services researchers experienced in engaging policymakers and health care managers argue that systematic reviews can be useful to these groups, and compiled a list of "myths" about the suitability of systematic reviews for use by decisionmakers.…”

“…50 Wye 2015 studied the use of evidence by local policymaking bodies in the U.K. and noted that policymakers preferred verbal over written presentation. 35 As described previously, health system decisionmakers often expressed a desire for more "conclusive" and "decisive" evidence. , and Munn 2014 53 described approaches that may address this perceived need by providing tools such as multicriteria decision analysis (MCDA), conjoint analysis, decision analytic modelling, or other quantitative approaches to clarifying value judgments and incorporating evidence.…”

“…Commissioners, "will always struggle to apply systematic reviews and randomized controlled trials, because paradoxically the very qualities that create such context-free, gold standard research render the studies less useful to commissioners." 35 Instead, "commissioners want "killer" stories that are rich in context." 35 In contrast, VHA executive level policymakers and their staff preferred RCTs, but also valued other types of evidence, such as case studies and patient preference studies.…”

“…35 Instead, "commissioners want "killer" stories that are rich in context." 35 In contrast, VHA executive level policymakers and their staff preferred RCTs, but also valued other types of evidence, such as case studies and patient preference studies. 36 Others also mentioned the need for local contextual information or applicability information, and descriptions of the uncertainty of the findings.…”

“…In general, they preferred summaries that were brief, presented in a "graded" format (key messages, executive summary) or presented verbally. 11,18,22,35 Two systematic reviews 49,50 addressed decisionmakers preferences for various derivative products based on systematic reviews. One found that decisionmakers described summary of findings tables with event rates and absolute differences as preferable.…”