Using systematic review methods to evaluate environmental public policy: methodological challenges and potential usefulness

Miljand, Matilda

doi:10.1016/j.envsci.2019.12.008

Cited by 29 publications

(19 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While these results are based on the experiences of EviEM, the findings are generalisable beyond EviEM as they concern the SR method itself, rather than the workings of EviEM. It should be noted that our conclusions are drawn from SRs within the natural science disciplines, but the issues are similar but even more complex had the SRs been carried within environmental social science (Miljand, 2020a). However, our results are confined to the environmental science and policy field, due to the very different institutional and professional context within for example medicine and social work where SRs are also frequently used.…”

Section: Resultsmentioning

confidence: 99%

“…They follow structured guidelines for how the review should be conducted, including how to select relevant literature and how to analyse the empirical data. This enables a more comprehensive description of the state-of-the-art in a field, new empirical conclusions to be drawn and theory to be developed (Miljand, 2020a). Traditional SR is guided by the idea of repeatability, objectivity and comprehensiveness (Miljand, 2020b), with the ‘aim to limit systematic error (bias) [in the literature review], mainly by attempting to identify, appraise and synthesise all relevant studies ’ (Petticrew and Roberts, 2006: 9).…”

Section: Introductionmentioning

confidence: 99%

“…It ran from 2012 until 2018 and published a total of 10 SRs and 4 systematic maps (SMs), mainly within natural sciences. An SM is ‘identical to traditional SR with the exception that is does not include any synthesis’ (Miljand, 2020b: 48). The interest in SR methods has grown since their introduction into the environmental field in the early 2000s.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Using systematic reviews to inform environmental policy-making

Miljand

Eckerberg

2022

Evaluation

Self Cite

View full text Add to dashboard Cite

There is a demand for scientific knowledge to make informed decisions in environmental policy. This study examines expectations of knowledge use, and how knowledge stemming from systematic reviews (SR) is being used through an analytical framework that distinguishes between instrumental, conceptual and legitimising evaluation use, as well as between process and product use. Empirically, we investigate knowledge generated from six SRs conducted through the Mistra Council for Evidence-based Environmental Management from the perspectives of those carrying out the SR and their targeted stakeholders. Our study reveals ways in which SRs are used and some characteristics that improve and some that hamper their usefulness. While the systematic method and the comprehensiveness of the SRs contribute positively to the usefulness, we found that the SRs produced were simultaneously too focused (lacking multiple perspectives), and too general (providing evidence on the effects of an intervention only at the general level) thereby restricting their usefulness. The time and resources it takes to produce an SR can also affect its usefulness compared to a traditional review.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using systematic reviews to inform environmental policy-making

Miljand

Eckerberg

2022

Evaluation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, the actual knowledge with respect to barriers and facilitators is based, above all, on the contributions of specific policy fields, where different types of reviews of the role of knowledge and evidence have been conducted. For instance, we can find many of these reviews in health policy (Morgan, 2010;Oliver et al, 2014;Parkhurst et al, 2018), environmental policy (Miljand, 2020), education policy (Tight, 2021;Zawacki-Richter et al, 2020) and criminal policy (Petrosino et al, 2018).…”

Section: Why This Metareview Is Needed and Helpfulmentioning

confidence: 99%

Mapping the use of knowledge in policymaking: barriers and facilitators from a subjectivist perspective (1990–2020)

Capano

Malandrino

2022

Policy Sci

View full text Add to dashboard Cite

The use of knowledge and evidence in policymaking is a recurrent topic of research due to its scientific and policy relevance. The existing and expansive body of literature has been scrutinised in various ways to grasp the dimensions of knowledge utilisation in policymaking, although most of this research has a monosectoral focus and is based on very general criteria of analysis that do not completely account for the complexity of policy making. This paper overcomes this limitation by enlightening the epistemological divide in the field between an objectivist and a subjectivist perspective and by distinguishing two different focuses in this literature: a focus on knowledge for policy making and a focus on knowledge in policy making. Based on this analytical distinction, the paper presents an original and unprecedented systematic, intersectoral metareview by considering the thirty-year period between 1990 and 2020 (approximately 1,400 were selected for fine-grained analysis). This metareview offers a broader and more detailed map with a clear idea of the distribution of interest in the topic among the different policy fields, a better classification of the theoretical/empirical content and research goals that scholars adopt and a novel and, above all, more fine-grained perspective on the types of conditions that favour or disfavour a significant role of knowledge in policymaking. Ultimately, and above all, this metareview identifies three highly relevant components of policy making that can facilitate or constrain the use of knowledge in policymaking more than others: values/ideology/beliefs, actors’ relationships, and policy capacities.

show abstract

“…Policy evaluation refers to producing qualitative and quantitative assessments of the value of formulated policies via certain procedures or techniques to check their effects in practice and determine whether to change or end them [ 1 ]. Policy evaluation is an indispensable link in the reasonable allocation of policy resources and scientific formulation of public policies [ 2 ], and plays a vital role in governments’ effective social intervention and social public management [ 3 , 4 ].…”

Section: Introductionmentioning

confidence: 99%

Quantitative Evaluation of Waste Separation Management Policies in the Yangtze River Delta Based on the PMC Index Model

Liu

2022

IJERPH

View full text Add to dashboard Cite

Numerous policies have been formulated and implemented to strengthen waste separation management activities in many countries. Waste separation management policies (WSMPs) must be evaluated as the precondition for reducing deviations from policy implementation and improving waste separation performance. Based on text mining technology and the construction of a policy modeling consistency (PMC) index model, we conducted a quantitative evaluation of 22 WSMPs issued by central governmental departments and provinces in the Yangtze River Delta, China from 2013 to 2021 and analyzed their optimization paths. The results suggest that the PMC index of the selected WSMPs has an upward trend. The average PMC index of 22 WSMPs was 6.906, indicating good quality in the policy texts. The PMC index identified seven, nine, five, and one of the policies as being perfect, excellent, good, and acceptable, respectively. The characteristics of WSMPs were further illustrated through PMC surface charts. Based on this, optimization paths for WSMPs with lower PMC indexes are proposed, which indicate that existing WSMPs have great potential for optimization in terms of harsher constraint regulations, context-appropriate incentives, and cultivation of market participants. Finally, this study provides a beneficial reference for similar cities or countries to improve their performance in the management of waste separation and environmental protection.

show abstract

Using systematic review methods to evaluate environmental public policy: methodological challenges and potential usefulness

Cited by 29 publications

References 46 publications

Using systematic reviews to inform environmental policy-making

Using systematic reviews to inform environmental policy-making

Mapping the use of knowledge in policymaking: barriers and facilitators from a subjectivist perspective (1990–2020)

Quantitative Evaluation of Waste Separation Management Policies in the Yangtze River Delta Based on the PMC Index Model

Contact Info

Product

Resources

About